L
L
libertyswift2016-02-14 21:05:37
Python
libertyswift, 2016-02-14 21:05:37

How to parse multiple pages by one xpath?

There are multiple url pages in the list
url1 = ' http://_______________________ '
url2 = ' http://_______________________ '
url3 = ' http://_______________________ '
url_list = ['url1','url2','url3','url4 ']
How to parse their data by one xpath
With one page,
from grab import Grab
g = Grab()
g.go(' http://______________________________ ')
print (g.doc.select('//div[@class ="______________"]').text()[1:])
And how to do it for a list of url where xpath doesn't change?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Dimonchik, 2016-02-14
@libertyswift

cycle no?

g = Grab()
for url in urls:
  g.go('http://______________________________')
  print (g.doc.select('//div[@class="______________"]').text()[1:])

L
libertyswift, 2016-02-14
@libertyswift

What am I doing wrong?
from grab import Grab
g = Grab()
url1 = 'http:________________________________'
url2 = 'http:________________________________'
url3 = 'http:________________________________'
url4 = 'http:________________________________'
urls = ['url1','url2','url3 ','url4']
for url in urls:
g.go(urls)
print (g.doc.select('//div[@class="______________"]').text()[1:])
_________________________________________________________________________________________________________________________________
I found this solution, everything works BUT print prints the values ​​in a string in square brackets and quotes, so ['Value', 'Value', 'Value'] how can I print them in a column?
from grab import Grab
g = Grab()
url1 = 'http:_________________'
url2 = http:_________________'
url3 = http:_________________'
url_list = [url1, url2, url3 ]
def page_handler(url):
resp = g.go(url )
data = g.doc.select('//div[@class="________________"]').text()[1:]
return data
all_data = [page_handler(url) for url in url_list]
print(all_data)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question