The idea was to extract important details of a professional from his/her website. We picked up "doctors” as one of the professions and mined the following information:
Name
Email
Location
Contact Number
Specialisation
The project was divided into two segments:
Extract information from web-pages which contain information of a single doctor
Extract information from web-pages where multiple doctors are mentioned and their emails, locations and specialisation mentions are intermixed.
Algorithm Used:
Distant Supervision - A technique used for learning relations between entities in a dataset based on heuristic rules. We wanted to learn which set of information (name, location, email, etc…) was related to which doctor/professional.
Knowledge Graphs - Construction knowledge graphs between the different entities of the data.
Tools Used:
DeepDive - Stanford’s tool for extracting information from the web.
Our work was published as a paper in the International Conference on Computational Intelligence and Data Science (ICCIDS 2018).
Pipeline for dealing with the first scenario - when there is only one doctor’s information on the webpage
Pipeline for dealing with the second scenario - when there are multiple doctors mentioned in the same webpage.