Automate web content collection

Communication department

Wiker is the first local network to connect all the players in medium-sized or rural towns via their economic, cultural or social information.

To do this, Wiker collects relevant information on territories from numerous Internet sources and aggregates it into a single service. This collection phase is time-consuming and error-prone.

Each member of staff spends an average of half an hour a day checking the websites they follow, picking up interesting information and labelling them.

Modules used

From the web to standardized database content

The solution

The SmartMyData platform was used to automate the entire process. The scraping worker retrieves the information from the websites concerned. The Function Worker cleans and formats the data, then the Prediction Worker labels the records. An INSEE module adds INSEE codes to the communes, and finally the Database worker saves the data in the database.

Source data and workflow




Web collect


Return on investment

Automate and label data, while minimizing the risks associated with human intervention.

Automatisation de la collecte d'informations sur une grande variété de sources

Automatic labeling of recordings via a Machine Learning module

Reallocation of employees to more rewarding tasks

Reduced risks associated with human intervention

Let's talk about your project!