The Manifesto Babel Machine

Introduction

The MANIFESTO Babel Project uses the Manifesto Project codebook’s values to identify the specific policy area or ideological stance for a text, ranging from foreign relations and economic policies to social issues, cultural values, and demographic considerations.

The codebook distinguishes 7 domains in the categories: external relations, freedom and democracy, political system, economy, welfare and quality of life, fabric of society, social groups.

The language models that the MANIFESTO Babel Machine uses were fine-tuned on training data containing only the Main-Categories of the coding scheme (the three-digit variables), excluding the Sub-Categories (the four-digit variables and the four digit variables with underscore). We use the label 0 to indicate that the given text contains no meaningful informations.

We have language-specific models for the following languages: Armenian, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Georgian, German, Greek, Hebrew, Hungarian, Icelandic, Italian, Japanese, Korean, Latvian, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Somali, Swedish and Turkish, but we encourage you to also submit datasets not covered under this list, as results may be useful for additional languages due to the nature of large language models.

You can upload your datasets here for automated MANIFESTO-coding. If you wish to submit multiple datasets one after another, please wait 5-10 minutes between each of your submissions. There are two possibilities for upload: pre-coded datasets or non-coded datasets. The explanation of the form and the dataset requirement is available here.

The upload requires to fill the following form on metadata regarding the dataset. We kindly ask you to upload your dataset, and in case of a pre-coded dataset, if available, please attach the codebook used besides the dataset.

The non-coded datasets should contain an id and a text column. The column names must be in row 1. You are free to add supplementary variables to the dataset beyond the compulsory ones in the columns following them.

Pre-coded datasets must contain the following columns: id, text, cmp_code. The column names must be in row 1. Uploading a pre-coded sample is optional, but it can help us with calculating performance metrics and fine-tuning the language model behind MANIFESTO Babel Machine. The detailed rules of validations are available here. The mandatory data format of cmp_code is numeric. You are free to add supplementary variables to the dataset beyond the compulsory ones in the columns following them. Automatic processing requires to follow these rules.

After you upload your dataset and your file is successfully processed, you will receive the MANIFESTO-coded dataset and a file (both in CSV format) that includes the three highest probability category predictions by the MANIFESTO Babel model and the corresponding probability (softmax) scores assigned to each label. Please be aware that interpreting softmax scores as absolute model confidences could lead to false assumptions about model performance.

If the files you would like to upload are larger than 1 GB, please reach out to us with the download link attached (such as Dropbox or Google Drive) using our contact form.

If you have any questions or feedback regarding the MANIFESTO Babel Machine, please let us know using our contact form. Please keep in mind that we can only get back to you on Hungarian business days.

Submit a dataset:

Processing Unit: GPU: Please use this for more than 1000 observations in individual files. CPU: Please use this for less than 1000 observations in individual files.

Name*

E-mail address*

Please use a valid e-mail address.

Institution name*

Dataset name*

Dataset language* -- Please Choose One -- Armenian Assamese Azerbaijani Basque Belarusian Bengali Bengali Romanize Bosnian Breton Bulgarian Burmese Burmese zawgyi font Cantonese Catalan Chinese (Simplified) Chinese (Traditional) Croatian Czech Danish Dutch English Esperanto Estonian Filipino Finnish French Galician Georgian German Greek Gujarati Hausa Hebrew Hindi Hindi Romanize Hungarian Icelandic Indonesian Irish Italian Japanese Javanese Kannada Kazakh Khmer Korean Kurdish (Kurmanji) Kyrgyz Lao Latin Latvian Lithuanian Macedonian Malagasy Malay Malayalam Mandarin Marathi Mongolian Nepali Norwegian Oriya Oromo Pashto Persian Polish Portuguese Punjabi Romanian Russian Sanskrit Scottish Gaelic Serbian Sindhi Sinhala Slovak Slovenian Somali Spanish Sundanese Swahili Swedish Tamil Tamil Romanize Telugu Telugu Romanize Thai Turkish Ukrainian Urdu Urdu Romanize Uyghur Uzbek Vietnamese Welsh Western Frisian Xhosa Yiddish

Please specify:

You selected a language for which limited or no training data is available. The prediction results, therefore, may be worse than for those with extensive training data: Armenian, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Georgian, German, Greek, Hebrew, Hungarian, Icelandic, Italian, Japanese, Korean, Latvian, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Somali, Swedish and Turkish.

Unit of observation* -- Please Choose One -- Quasi sentence Sentence Paragraph Full text Budget item

Period (from)*

Period (to)*

Level of dataset* -- Please Choose One -- Supranational/International State Substate

Geographical unit*

Use case* -- Please Choose One -- Research Commercial use Personal use Other

Please specify:

Description

The non-coded datasets should contain an id and text column. The column names must be in row 1. You are free to add supplementary variables to the dataset beyond the compulsory ones in the columns following them. All datasets must be uploaded in a CSV file format with UTF-8 encoding.

• Dataset in .CSV format:
• Choose file

• Pre-coded dataset:
• Choose file

• Codebook:
• Choose file

I have read the instructions and rules.

By ticking the checkbox I declare that this data does not contain any personal data and I am responsible for the legitimacy of the origin of the data. The uploaded data is suitable for text mining and I voluntarily provided the uploaded file(s) to poltextLAB. I also grant permission to poltextLAB researchers to make the data visualisation of the data I uploaded publicly available. In addition, I agree that the uploaded data may be stored on servers within the European Union and be used for data analysis and training purposes without time limit.

Loading...

Troubleshooting

If you are experiencing problems with the upload form, or your submission returns an error message (particularly "Something unexpected happened during upload. Please try again later."), please try performing the following steps:

• If you use an adblocker browser extension, please turn it off for our site. Adblockers may interfere with legitimate functionality, such as the dropdowns on the upload form. (We do not serve ads on the site.)
• Try turning off your VPN.
• Try submitting your data from another browser, preferably with default settings.

If you are still receiving the "Something unexpected..." error message, please get in touch with us via our email address or the contact form. Try to add as much information as possible, e.g., what browser you are using, notable browser extensions, whether you are using a VPN or not, and exactly how you tried to submit the data (for example, you filled out everything but waited 10 minutes before pressing submit).

The research was supported by the Ministry of Innovation and Technology NRDI Office within the RRF-2.3.1-21-2022-00004 Artificial Intelligence National Laboratory project and received additional funding from the European Union's Horizon 2020 program under grant agreement no 101008468. We also thank the Babel Machine project and HUN-REN Cloud (Héder et al. 2022; https://science-cloud.hu) for their support.

HOW TO CITE: If you use the Babel Machine for your work or research, please cite this paper:
Sebők M, Kacsuk Z. The Multiclass Classification of Newspaper Articles with Machine Learning: The Hybrid Binary Snowball Approach. Political Analysis. 2021;29(2):236-249. doi:10.1017/pan.2020.27