📄 Masader Plus: A New Interface for Exploring +500 Arabic NLP Datasets

Summary

Arabic NLP has produced a large number of datasets, but finding, comparing and reusing them is hard. Earlier listings were either literature reviews or plain catalogues that could not be searched in a useful way. The authors build on the Masader metadata catalogue (Alyafeai et al., 2021) and add Masader Plus, an open-source web interface released under GPL-3.0 that covers more than 500 Arabic NLP datasets.

The system has four parts:

• a Flask backend on Heroku that pulls updated metadata every ten minutes and caches it in Redis;
• semantic clustering, where Sentence Transformers embeddings (all-MiniLM-L6-v2, served through the HuggingFace Inference API) are grouped with K-Means;
• a small REST API whose filter expressions use the Pandas query language;
• a frontend with a filtering sidebar, per-dataset data cards, metadata charts and a contribution form.

Community work was central to growing the catalogue. A hackathon organised with HuggingFace added 145 datasets. The resource matters because a structured, filterable and living inventory turns a scattered literature into something researchers can actually navigate and maintain.

Key points

• The API offers four endpoints: /datasets/schema lists the fields, /datasets filters records and selects fields, /datasets/[index] returns one record, and /datasets/tags returns the distinct values of chosen fields.
• Embeddings are computed from each dataset's Name, Description and Abstract, then clustered with K-Means. On the Stats page, these clusters are plotted next to charts covering dialects, scripts, forms, tasks, domain, venues, hosts, access, licenses, ethical risks and year.
• Users can filter by task (the 20 most frequent tasks are shown first), by dialect (region or country), by access level, by license and by year range. The catalogue currently spans 2000 to 2022.
• The hackathon took place on 10 June 2022 and stayed open for edits until 20 June. It added 145 datasets, and over 100 people joined the project's Discord within one week. For comparison, HuggingFace Datasets held fewer than 100 Arabic datasets on 1 August 2022.
• The new Google Form gives each field an explanation and offers drop-down choices for multi-option fields, replacing the older two-spreadsheet process that was error-prone. Contributors are told the status of their submission, and administrators review entries before they go live.

Interesting ideas

• I1. Exposing a tags endpoint that returns every distinct value of a field makes it easy to build filters and to spot inconsistent labels. — For Mova (tree): A metadata catalogue of our UD treebanks (genre, UD version, license, annotation dialect), with distinct-value listing, would help us assemble training mixes and catch mismatched dialect labels before conversion.
• I2. Replacing a free-text spreadsheet with a form that explains every field and limits options to fixed choices cuts entry errors, and a reviewer approval step keeps the final catalogue clean. — For Mova (world): Compiled knowledge entries such as measures, verb-class additions or ontology links could be entered through typed forms with fixed value sets and a review gate, so the knowledge graph never receives malformed facts.
• I3. Every catalogue item gets its own card with a button to report wrong or missing information, either anonymously or through GitHub, which turns users into continuous auditors. — For Mova (answer): Each answered test item or fable QA output could link to an explainable card with a report option, feeding flagged errors back as cases for repair.
• I4. Derived information (embeddings, clusters) is refreshed on a schedule and cached, while a public changelog records what changed and who changed it. — For Mova (learn): Silver data and self-induced repair rules could be rebuilt periodically with a changelog naming the witness or teacher behind each change, giving clear provenance when evaluation scores shift.
• I5. Clustering resources by the meaning of their descriptions gives a map of the collection that shows crowded areas and empty ones. — For Mova (learn): Grouping our evaluation and silver sets by description would reveal redundant sets and missing domains, guiding where to add new tests or negative controls.