Quantcast
Channel: Active questions tagged selenium - Stack Overflow
Viewing all articles
Browse latest Browse all 97830

Web Scraping WeatherUnderground

$
0
0

I'm doing the project that reads the data every hour from WeatherUnderground and it saves the data into a txt file

from selenium import webdriverfrom selenium.webdriver.common.by import Byfrom datetime import datetimedef scrape_weather_data(url, time_selector, temperature_selector, humidity_selector, precip_selector, wind_speed_selector, pressure_selector, amount_selector):driver = webdriver.Chrome()  driver.get(url)  # Enter The website# Generate text file name based on day and hourcurrent_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")filename = f'dane_pogodowe_{current_time}.txt'# Saving data into a .txt filewith open(filename, 'a') as file:    time_elements = driver.find_elements(By.CSS_SELECTOR,time_selector)    temperature_elements = driver.find_elements(By.CSS_SELECTOR,temperature_selector)    humidity_elements = driver.find_elements(By.CSS_SELECTOR,humidity_selector)    precip_elements = driver.find_elements(By.CSS_SELECTOR,precip_selector)    wind_elements = driver.find_elements(By.CSS_SELECTOR, wind_speed_selector)    pressure_elements = driver.find_elements(By.CSS_SELECTOR, pressure_selector)    amount_elements = driver.find_elements(By.CSS_SELECTOR, amount_selector)    for i in range(len(time_elements)):        time = time_elements[i].text.strip()        temperature = temperature_elements[i].text.strip()        humidity = humidity_elements[i].text.strip()        precip = precip_elements[i].text.strip()        wind = wind_elements[i].text.strip()        pressure = pressure_elements[i].text.strip()        amount = amount_elements[i].text.strip()        file.write(f"Czas: {time}\n")        file.write(f"Temperatura: {temperature}\n")        file.write(f"Wilgotność: {humidity}\n")        file.write(f"Szansa opadów: {precip}\n")        file.write(f"Wiatr: {wind}\n")        file.write(f"Ciśnienie: {pressure}\n")        file.write(f"Ilość opadów: {amount}\n")        file.write("\n")    file.write("------------------------------\n")driver.quit()  # Close the Web browserprint("Dane pogodowe zostały zapisane.")# URLurl = 'https://www.wunderground.com/hourly/pl/kucoby'# Selectors CSStime_selector = '#hourly-forecast-table > tbody > tr:nth-child(1) > td.mat-cell.cdk-cell.cdk-column-timeHour.mat-column- timeHour.ng-star-inserted > span'temperature_selector = '#hourly-forecast-table > tbody > tr:nth- child(1) > td.mat-cell.cdk-cell.cdk-column-temperature.mat- column-temperature.ng-star-inserted'humidity_selector = '#hourly-forecast-table > tbody > tr:nth- child(1) > td.mat-cell.cdk-cell.cdk-column-humidity.mat-column- humidity.ng-star-inserted'precip_selector = '#hourly-forecast-table > tbody > tr:nth- child(1) > td.mat-cell.cdk-cell.cdk-column-precipitation.mat- column-precipitation.ng-star-inserted'wind_speed_selector = '#hourly-forecast-table > tbody > tr:nth- child(1) > td.mat-cell.cdk-cell.cdk-column-wind.mat-column- wind.ng-star-inserted'pressure_selector = '#hourly-forecast-table > tbody > tr:nth- child(1) > td.mat-cell.cdk-cell.cdk-column-pressure.mat-column- pressure.ng-star-inserted'amount_selector = '#hourly-forecast-table > tbody > tr:nth- child(1) > td.mat-cell.cdk-cell.cdk-column- liquidPrecipitation.mat-column-liquidPrecipitation.ng-star- inserted'# Calling a function and saving the Datascrape_weather_data(url, time_selector, temperature_selector, humidity_selector, precip_selector, wind_speed_selector, pressure_selector, amount_selector)

And the data reading is based on CSS_Selectors as you can tell, but the problem is that when it enters the website, the default setting is that it opens in the imperial system, not the metrical that i use. And i can't really convert it mathematically into metric system, due to the fact that the variables are saved as strings. There is a option on this website to convert data from imperial into metrical system, but to be fair i have no idea, how to implement it.

EDIT:

I found out the key that is responsible for setting the metric system:


Viewing all articles
Browse latest Browse all 97830

Trending Articles