Visualizing every job in the world
Imagine you have to classify every single job title in the world into 10 categories, how would you go about it?
This is a fairly hard problem to solve. However, the European Union has actually taken it on. They named it: the ESCO project. (ESCO stands for European Skills, Competences, Qualifications and Occupations)
Damn I love the EU.
What if we scraped their database of 3001 occupations and ran them through the famous BERT model? We could look at all the alternative names for each title/occupation, and build a network of jobs.
The result is this projection: a network of every job in the world, that clusters titles by similarity.
You can also run the PCA or t-SNE algorithms through it. These will cluster the jobs in slightly different ways. Go ahead and look for your job title on the right pane, and see the most similar jobs out there (at least according to BERT).
February 10, 2022