How to Scrape Job Aggregator Websites Like Naukri.com using BeautifulSoup and Python?

pip install requests pip install beautifulsoup4
import requests from selenium import webdriver from bs4 import BeautifulSoup import time import pandas as pd
url = "https://www.naukri.com/financial-analyst-jobs-in-mumbai?k=financial%20analyst&l=mumbai" page = requests.get(url) page.text
driver = webdriver.Chrome("D:\\Selenium\\chromedriver.exe") driver.get(url) time.sleep(3) soup = BeautifulSoup(driver.page_source,'html5lib') print(soup.prettify()) driver.close()
df = pd.DataFrame(columns=['Title','Company','Ratings','Reviews','URL'])
results = soup.find(class_='list')
job_elems = results.find_all('article',class_='jobTuple bgWhite br4 mb-8')

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store