vb.net - Web scraping with a pop-up (Visual Basic) -


i'm trying scrape page, when login page displays pop before page need (welcome blah-blah-blah...don't hit refresh slow process...etc.etc...).

naturally, httpwebrequest scrapes data , not page follows.

the popup self cancels if httpwebrequest wait second or 2 , scrape, work - or - if can 2 scrapes (and discard 1st one) in same session work too.

here's code:

dim cookiejar new cookiecontainer dim request httpwebrequest = webrequest.createhttp(textbox1.text) request.cookiecontainer = new cookiecontainer() request.cookiecontainer.add(new uri(textbox1.text),           new cookie("id", "1234")) request.preauthenticate = true request.credentials = credentialcache.defaultcredentials request.useragent = "user-agent: mozilla/5.0 (windows nt 6.1; wow64)" request.allowautoredirect = true request.maximumautomaticredirections = 4 request.maximumresponseheaderslength = 4  dim response webresponse = directcast(request.getresponse(), httpwebresponse)  dim webresult string = new streamreader(response.getresponsestream()).readtoend() textbox2.text = webresult

thanks in advance suggestions.


Comments

Post a Comment

Popular posts from this blog

javascript - Clear button on addentry page doesn't work -

this addentry blog page have made scratch , there underlying problems code particularly clear button supposed clear text entered when button clicked on doesn't work have saved html file problem. also 1 other problem submit button works doesn't pass of text index file have written php , reason whenever click on submit button redirects me index page without posting entry have made. <!doctype html> <html> <head> <script> function message() { alert("clear form?"); } </script> <style> body {background-color: beige;} h1 {color: black;} p {color: black;} textarea { width: 50%; height: 150px; padding: 12px 20px; box-sizing: border-box; border: 2px solid #ccc; border-radius: 4px; background-color: white; font-size: 16px; resize: none; } div { padding-left: 20px; } </style> </head> <meta charset="utf-8"> <link rel="stylesheet" href="....
Read more

python - Error: Unresolved reference 'selenium' What is the reason? -

# -*- coding: utf-8 -*- selenium import selenium import unittest, time, re class rc(unittest.testcase): def setup(self): self.verificationerrors = [] self.selenium = selenium("localhost", 4444, "*chrome", "http://stackoverflow.com/") self.selenium.start() def test_rc(self): sel = self.selenium sel.open("/") sel.type("id=kw", "selenium") sel.click("id=container") sel.click("id=su") def teardown(self): self.selenium.stop() self.assertequal([], self.verificationerrors) if __name__ == "__main__": unittest.main() this recorded selenium ide, selenium remote control code. in ide test passed.   in pycharm open, first line ( from selenium import selenium ) there error: unresolved reference 'selenium' what reason? my environment is python3.5 selenium3.3.1 selenium-server-standalone-...
Read more

asp.net ajax - Jquery scroll to element just goes to top of page -

i have jquery updates page content via ajax (paging through sub table content) , want page "jump" (scroll) top of section content has been updated. found code try , perform scroll ever goes right top of page , not specific element : <script type="text/javascript"> $(document).ready(function() { $(".page-number").on("click", function() { var page = parseint($(this).html()); $.ajax({ url: '@url.action("productreview")', datatype: 'html', data: { "id": @model.product.productid, "page": page }, success: function (data) { $("#review-list").html(data); $("#page-number-"+ page).addclass("selected"); //location.hash = "#prodreviews"; ...
Read more