Quantcast
Channel: Active questions tagged selenium - Stack Overflow
Viewing all articles
Browse latest Browse all 97813

Scraping the moves of Lichess in BS4

$
0
0

i would like to create an Suggestion AI about a best move in chess. I dont really get the html tag of lichess in bs4.

In Chrome I can actually see the Tag in bs4 it is an script based command. I will show some code because for a better understanding.

Thats my script for scraping the data.

from bs4 import BeautifulSoup
from requests import get
from time import time


def scrape_data_from_lichess(url):
    response = get(url)
    html_soup = BeautifulSoup(response.text, 'html.parser')
    move_played = html_soup.find_all('move')
    print(move_played)

while not scrapeData:
    scrape_data_from_lichess(webpage + url)
    time.sleep(10)

That is my Google Devtool respond of the location of the tag. It is really odd because the Chrome Extension 'Web Scraper' can actually detect these tags.

<div class="moves">
  <index class="">1</index>
  <move class="">d4</move>
  <move class="">d6</move>
  <index class="">2</index>
  <move class="active">Nf3</move>
  <move class="">Nf6</move>
</div>

And the last one is my bs4 result of the html text.It is the short version. I modified it.

LichessRound.boot({
    data: {"game":{"player":"white","opponent":{"color":"black","name":null,"ai":1},"san":"d4","san":"d6","san":"Nf3",{"san":"Nf6"}]},

Here is the long version.

LichessRound.boot({
data: {"game":{"id":"4PAVr8QI","variant":{"key":"standard","name":"Standard","short":"Std"},"speed":"correspondence","perf":"correspondence","rated":false,"initialFen":"rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1","fen":"rnbqkb1r/ppp1pppp/3p1n2/8/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - 2 3","player":"white","turns":4,"startedAtTurn":0,"source":"ai","status":{"id":20,"name":"started"},"createdAt":1536082009777,"lastMove":"g8f6","opening":{"eco":"A46","name":"Indian Game: Wade-Tartakower Defense","ply":4}},"clock":null,"correspondence":null,"player":{"color":"white","name":null,"user":{"id":"gamelinetv","username":"Gamelinetv","online":true,"perfs":{"correspondence":{"games":0,"rating":1500,"rd":350,"prog":0,"prov":true}}},"rating":1500,"provisional":true,"version":4,"spectator":true},"opponent":{"color":"black","name":null,"ai":1},"orientation":"white","url":{"socket":"/4PAVr8QI/white/socket","round":"/4PAVr8QI/white"},"pref":{"animationDuration":300,"coords":2,"replay":2,"clockTenths":1,"clockBar":true,"highlight":true,"destination":true,"rookCastle":true,"showCaptured":true},"evalPut":false,"steps":[{"ply":0,"uci":null,"san":null,"fen":"rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"},{"ply":1,"uci":"d2d4","san":"d4","fen":"rnbqkbnr/pppppppp/8/8/3P4/8/PPP1PPPP/RNBQKBNR b KQkq - 0 1"},{"ply":2,"uci":"d7d6","san":"d6","fen":"rnbqkbnr/ppp1pppp/3p4/8/3P4/8/PPP1PPPP/RNBQKBNR w KQkq - 0 2"},{"ply":3,"uci":"g1f3","san":"Nf3","fen":"rnbqkbnr/ppp1pppp/3p4/8/3P4/5N2/PPP1PPPP/RNBQKB1R b KQkq - 1 2"},{"ply":4,"uci":"g8f6","san":"Nf6","fen":"rnbqkb1r/ppp1pppp/3p1n2/8/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - 2 3"}]},

I know that Selenium can fix the problem. But it start a new Browser and I just want to paste the script and get the data of the moves. So I decided to use BS4 but I am open for new ideas.

Hopefully you guys and girls can help me.

Best Regard Cren


Viewing all articles
Browse latest Browse all 97813

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>