Data Collection

3 billion articles as an information base

An accurate and reliable analysis of political risks does not – contrary to the widely shared assumptions – require secret information from intelligence services, but detailed and dependable sources of information, which the client can check independently. Via co-operations with information providers and the application of a specifically developed software, more than three billion articles p.a. or 40,000 sources, including newspapers, online journals, blogs and social media are searched for relevant information by the CONIAS algorithms. After this phase, ca. 250,000 relevant articles p.a. remain, which are subsequently evaluated and converted into approximately 180,000 measures p.a., which then enter the database. Several measures even have two or more referenced source articles. As a result of this process, CONIAS currently monitors 500 ongoing political conflicts worldwide.

High quality data collection within a short space of time

The collection of sensitive data for conflicts and security is time consuming and requires particular diligence. Based on our 20 years of experience and the latest specifically developed software, we managed to reduce the time requirement compared to previous methods by 80 % and increased the quality.

The classification and collection of data during the survey phase is executed via a query term optimized keyword search of more than 40,000 different sources or three billion printed and online articles worldwide p.a. Subsequently, all articles are screened by a self-improving algorithm and audited by the coders. During the acquisition phase all relevant articles are then reviewed semi-automatically. The results are amended or, if necessary, determined new by the coders. This process is based on a codebook that contains considerably more than 200 distinct measures and allows for the conducted activities to be transmitted into evaluable data sets. To ensure the relevance and reliability of these datasets, they are compared with already processed entries, outliers are marked, and the entire data is revised and supplemented by scientific co-operation partners from the corresponding regions. Only after a final checkup by an editor the data is transferred into the CONIAS Risk Intelligence system and used for assessment and analysis for customers.

Experience and advanced software technology coupled with a large network of professional co-operation partners, enables us to quickly update our data by simultaneously applying meticulous care. These updates are executed routinely on a monthly basis. However, in special periods of crisis, a daily risk-update can be supplied.