让 Python 在下载的时候使用 IPv4 域名解析 - Jacky Liu's Blog
让 Python 在下载的时候使用 IPv4 域名解析
---- 这个话题由来于我想从深交所的网站扒些信息下来,发现用 Python 下载时,每次都要 30 秒才能连上,浏览器访问却正常。于是查找原因,最后用抓包分析,发现问题出在域名解析阶段。浏览器访问时,向 DNS 服务器发送的是 A 型查询,要求做 IPv4 地址解析; 而 Python 的 urllib.request 模块发送的却是 AAAA 型查询,要求做 IPv6 地址解析,对深交所网址进行 IPv6 解析居然要花 30 秒,不知道为什么。
---- 然后就是各种 Google + 查看代码,追根溯源。不细说了,只贴结果:
def Resolver_IPv4(hostname): return socket.gethostbyname(hostname) # 此函数返回 IPv4 地址 class HTTPConnection_IPv4(http.client.HTTPConnection): def connect(self): self.sock= socket.create_connection((Resolver_IPv4(self.host), self.port), self.timeout) class HTTPHandler_IPv4(urllib.request.HTTPHandler): def http_open(self, req): return self.do_open(HTTPConnection_IPv4, req) urlopen_IPv4= urllib.request.build_opener(HTTPHandler_IPv4).open
如此以后,用这个 urlopen_IPv4() 函数来取代平常用的 urllib.request.urlopen() 函数,就可以用 IPv4 地址解析的方式来下载:
webfile= urlopen_IPv4(fullurl='http://www.szse.cn/', timeout=3)
---- 最后说一下,抓包感觉就跟作弊一样,爽!
2018年10月25日 10:05
于是查找原因,最后用抓包分析,发现问题出在域名解析阶段。浏览器访问时...
2022年9月07日 21:04
Here we give the Samagra question pool questions in PDF format. This help students with their reference. We provide the PDF of all chapters in a single file. Social Science, Biology, Chemistry, Maths questions in all chapters are available. Samagra Question Pool This PDF note is also helpful for the Kerala PSC exam. The new questions from Kerala PSC are from Samagra questions. Class 10 Samagara Question Pool is below. You can easily download it as a PDF file. The Samagra portal aims to give the study materials in digital form. Below we give the Samagra question pool. You can easily download or read the notes from here.