selenium跳过webdriver检测并模拟登录淘宝_python

来源:脚本之家  责任编辑:小易  

www.zgxue.com防采集请勿采集本网。

简介

模拟登录淘宝已经不是一件新鲜的事情了,过去我曾经使用get/post方式进行爬虫,同时也加入IP代理池进行跳过检验,但随着大型网站的升级,采取该策略比较难实现了。因为你使用get/post方式进行爬取数据,会提示需要登录,而登录又是一大难题,需要滑动验证码验证。当你想使用IP代理池进行跳过检验时,发现登录时需要手机短信验证码验证,由此可以知道旧的全自动爬取数据对于大型网站比较困难了。

selenium是一款优秀的WEB自动化测试工具,所以现在采用selenium进行半自动化爬取数据,支持模拟登录淘宝和自动处理滑动验证码。

编写思路

由于现在大型网站对selenium工具进行检测,若检测到selenium,则判定为机器人,访问被拒绝。所以第一步是要防止被检测出为机器人,如何防止被检测到呢?当使用selenium进行自动化操作时,在chrome浏览器中的consloe中输入windows.navigator.webdriver会发现结果为Ture,而正常使用浏览器的时候该值为False。所以我们将windows.navigator.webdriver进行屏蔽。

在代码中添加:

options = webdriver.ChromeOptions() # 此步骤很重要,设置为开发者模式,防止被各大网站识别出来使用了Selenium options.add_experimental_option('excludeSwitches', ['enable-automation']) self.browser = webdriver.Chrome(executable_path=chromedriver_path, options=options)

同时,为了加快爬取速度,我们将浏览器模式设置为不加载图片,在代码中添加:

options = webdriver.ChromeOptions() # 不加载图片,加快访问速度 options.add_experimental_option("prefs", {"profile.managed_default_content_settings.images": 2})

至此,关键的步骤我们已经懂了,剩下的就是编写代码的事情了。在给定的例子中,需要你对html、css有一定了解。

比如存在以下代码:

self.browser.find_element_by_xpath('//*[@class="btn_tip"]/a/span').click() taobao_name = self.wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, '.site-nav-bd > ul.site-nav-bd-l > li#J_SiteNavLogin > div.site-nav-menu-hd > div.site-nav-user > a.site-nav-login-info-nick '))) print(taobao_name.text)

第1行代码指的是从根目录(//)开始寻找任意(*)一个class名为btn_tip的元素,并找到btn_tip的子元素a标签中的子元素span

第2行代码指的是等待某个CSS元素出现,否则代码停留在这里一直检测。以.开头的在CSS中表示类名(class),以#开头的在CSS中表示ID名(id)。A > B,指的是A的子元素B。所以这行代码可以理解为寻找A的子元素B的子元素C的子元素D的子元素E出现,否则一直在这里检测。

第3行代码指的是打印某个元素的文本内容

使用教程

点击这里下载下载chrome浏览器

查看chrome浏览器的版本号,点击这里下载对应版本号的chromedriver驱动

pip安装下列包

[x] pip install selenium

点击这里登录微博,并通过微博绑定淘宝账号密码

在main中填写chromedriver的绝对路径

在main中填写微博账号密码

#改成你的chromedriver的完整路径地址 chromedriver_path = "/Users/bird/Desktop/chromedriver.exe" #改成你的微博账号 weibo_username = "改成你的微博账号" #改成你的微博密码 weibo_password = "改成你的微博密码"

演示图片

源代码

项目源代码在GitHub仓库

项目持续更新,欢迎您star本项目

Abbe numberabrasionabrasive wheelsacid etchingacid polishingacid stampingacoustic insulationadsorptionagate glassair trap,air lockair twistair-linesalarm glassalkalialkali-borosilicate glassalkaline earthsaluminaalumina-silicate glassamber glassampoulesandalusiteanhydriteannealingantenna glassantimonyantique glassarmour plate glassarsenicasbestosautoclaveaventurine glassAZS refractoriesbadgingbag filterball gathererbandingbariumbarium carbonatebarium crown glassbarium flint glassbarium oxidebarium-borate glassbarsbatbatchbauxitebeads,glassbelt grindingbendingbevel polishingbevellingbird swingbirdcageblankblistersblow-and-blow processblowpipeboraxboric acidboric oxideboron compoundsborosilicate glassbottle-making machinebottlesbottom plateBoudin processbronze alloybubblerbubblesbulletproof glassbullionsburn-offburnersbushingsbutylcadmium sulphidecalciacalcined aluminacalcium carbonatecalcium fluoridecalcium oxidecalcium sulphatecalmecalumitecar rear-view mirrorcarboyscarvingcased glasscasingcast glasscast ironcast iron alloycastablecastingcaustic sodacentrifuging processceramizing processcerium oxidechairchalcogenide glasschina claychromitechromiumclassifierclimatic cabincoatingcobaltcold endcolemanitecolourcomposition of glassconditioningconductive coatingcontainerscontainers,coatingcontainers,defectscontainers,enamelingcontainers,formingcontainers,inspectioncontainers,quality controlcontainers,surface treatmentcontinuous processes,tankconveyorconveyor,beltconveyor,crosscooling,coolantcopper oxidecopper-wheel engravingcords(defect in glass)corecore formingcorning special glasscorrosion of glasscrack detectioncrack-offcrackle glasscrizzling,crisselingcrown glasscrownscruciformcryolitecrystal glasscrystallisationculletcullet separationcuttingcylinder glassdalle glassDanner processday tankDCL techniquedebiteusedecolorisingdecolourisingdecorationdedustingdeionizingdeleting devicedensity,of batch materialsdensity,of glassdensity of refractoriesdepalletiserdeposition of films from solutiondesiccant saltdessicant saltdevitrificationdewar flaskdiamond air trapdiamond tooldiamond,wheelsdielectric loss factordimensional gaugingdimensional tolerancedirect screen processdistortiondoghousedolomitedomestic waredosing systemdouble glazingdouble-edgedrawing,rod or tubedrawn glassdrillingdrillsdrive systemdrying ovendrying sandDuPont coatingdurabilityedge-meltingedgingelectrodeembossingenamelengravedengravingetched photosensitive glass-ceramicexpansionexpansion of tank blockextrudingextrusionfacade glazing/wallingfeederfeldsparfelsparfelt wheelsfibre glassfigured glassfilm,deposition from solutionfilm,vapour depositionfiningfinishedfire polishingfire-resisting glassfireclayfiringflat glassflat glass,enamellingflat glass,inspectionflat glass,surface treatmentflatteningfloat processfluefluorescent lampfluoridefluorinefluoritefluorsparfluxfoam glassforehearthformingforming,containerforming,flatforming,glass fibreforming,optical glassforming,rodforming,tubingforsteritefoundingfourcault processfriction coefficients of coating materialsfritfrostingfurnacefurnace,bottomfurnace,crownfurnace,direct-fired continuous tankfurnace,electricfurnace,operationfurnace,pochetfurnace,potfurnace,sidewallsfurnace,superstructurefurnace,tankfurnace,temperaturefused silica/quartzfused silica/quartzfusingfusing glass-to-glassfusion-cast tank blockgas fillinggathergathering irongauging equipmentGeneva wheelGiammarco-Vetrocoke processgildingglassglass,compositionglass,flowglass level controlglass-ceramic to metal sealingglass-ceramicsglass-fiber-reinforced concreteglass-fiber-reinforced plasticglory holegobgob distributorgob feedergob weight controlgoldgradinggraphitegreen glassgreenhouse glassgrindinggypsumhalogen incandescent lamphandlingheat exchangerheat resistant glassheat-resistant glassheating uphollow warehomogeneity of glasshot extrusionhot melthot spothydrated aluminahydrofluoric acidI.S.machineIGimpact testingincandescent lamp bulbinfrared lampinfrared thermometerinfrared-absorbing glassinfrared-transparent glassinjection equipmentinleakageinspectioninstallationinsulatinginsulating,to exclude air inleakageinsulating,to retain heatinsulator,high-voltageinternal pressure testingion-exchange,strengthening byiridiumironiron bowlsiron oxideisostatic pressingkilnkiln formingKoppers-Totzek processKorolyov processkyanitelabellinglaboratory testinglacquerlaminated glasslamplampworkinglanthanum glasslaser cuttinglaser glasslathesleaching of glassleadlead crystallead oxideslead silicateLeblanc processlehrlevel controlLibbey-Owens processlight ashlightweightinglimelimesparlimestonelininglinishinglithiumlow emissivity glasslow-ElubricantLurgi processlustreLynch machinemachines,formingmagnesiamagnesitemagnesium carbonatemanganese dioxidemanglemarkingmarveringmechanical properties of refractoriesmeltmeltingmelting endmelting points of batch materialsmetallizingmetalsmillingmineral fibremirrormixer feedmixing,batchmoilmolybdenummonofraxmosaic glassmouldmould,blankmould,blockmould,blowmould,cast ironmould,dipmould,lubrication ofmould,opticmould,pastemould,pre-heating ovenmould,steelmuffle kilnmullitemycrodyne processnarrow-neck warenatural gasneck-ringneodymiumnepheline syenitenickel oxidenitratesnitrogenNNPB processONeill machineobsidianopal glassoptical glassoptical thickness gaugeOwens-Illinois coating techniquesoxygenpackagingpackingpalladiumpantographparisonpatterned glasspelletpelletisingphialPhilips processphosphate glassphotochemical forming processphotosensitive glasspick-up decorationPilkington double-pass wired glass processPilkington double-pass wired processPittsburgh processplate glassplatinum feederplungerpolishingpolishing wheelspollution controlpolysulphidepolyurethanepontil,pontil markporositypotpot archpotashpotassium bicarbonatepotassium carbonatepotassium dichromatepotassium nitratePPG-Pennvernon processpre-heatingprecious metalspress-and-blow processpress-blow processpressed glasspressed warepressingPreston testprofilepush-uppusher systemputtyPVBpyrex glasspyrometerrare earth elementsrasoriteraw materials,for basic refractoriesraw materials,for glassraw materials,for tank blocksrecuperative heatingrecuperative tankrecuperatorsredoxredox equilibriarefiningrefractive indexrefractoriesrefractories,chemical propertiesrefractories,chrome-aluminarefractories,chrome-magnesiterefractories,concreterefractories,fusion-castrefractories,insulatingrefractories,monolithicrefractories,(non-fusion cast)refractories,physical propertiesrefractories,silicarefractory claysregenerative heatingregenerative tankresin laminatingrhodiumrim finishingrodrolled glassrotating tablerutheniumsafety glasssaltcakesandsand blastingsawingscraper,hot glassscreen printingsealing,insulating glasssealing,stemwareseam markseedseleniumshear bladesshearssheet glass processesshrink wrap packagingsilicasilica sandsilicone extruderssiliconessilkscreeningsillimanitesilversilveringSindanyo?sizeslagslug bottomsodasoda ashsoda-lime glasssodalime glasssodium boratesodium carbonatesodium chloridesodium discharge lampssodium fluoridesodium hydroxidesodium nitratesodium silicatesodium silicofluoridesoftening temperaturesol-gel processsolar control glassSolvay processspacerspikesspinning,hollow warespraying systemsputteringstackerstaple fibrestemwarestipplingstirrerstonesstraight-edge engravingstraight-edgingstrontium oxidesuck-blow processsuction cupsulphatessulphuringsurface..内容来自www.zgxue.com请勿采集。


  • 本文相关:
  • 使用selenium模拟登录解决滑块验证问题的实现
  • python使用selenium模块模拟浏览器抓取斗鱼直播间信息示例
  • python使用selenium模块实现模拟浏览器抓取淘宝商品美食信息功能示例
  • selenium 模拟浏览器动态加载页面的实现方法
  • 浅谈python爬虫使用selenium模拟浏览器行为
  • python编程使用selenium模拟登陆淘宝实例代码
  • 利用selenium 3.7和python3添加cookie模拟登陆的实现
  • selenium自动化模拟键盘快捷键操作实现解析
  • python走楼梯问题解决方法示例
  • pandas pivot_table() 按日期分多列数据的方法
  • python基于itchat模块实现微信防撤回
  • 利用scrapy将爬到的数据保存到mysql(防止重复)
  • python 3.5下xadmin的使用及修复源码bug
  • python脚本执行cmd命令并返回结果的例子
  • python实现的服务器示例小结【单进程、多进程、多线程、非阻塞式
  • python生成九宫格图片
  • python使用while循环花式打印乘法表
  • 一个基于flask的web应用诞生 用户注册功能开发(5)
  • 玻璃制造行业的英语
  • 网站首页网页制作脚本下载服务器操作系统网站运营平面设计媒体动画电脑基础硬件教程网络安全vbsdos/bathtahtcpythonperl游戏相关vba远程脚本coldfusionruby专题autoitseraphzonepowershelllinux shellluagolangerlang其它首页使用selenium模拟登录解决滑块验证问题的实现python使用selenium模块模拟浏览器抓取斗鱼直播间信息示例python使用selenium模块实现模拟浏览器抓取淘宝商品美食信息功能示例selenium 模拟浏览器动态加载页面的实现方法浅谈python爬虫使用selenium模拟浏览器行为python编程使用selenium模拟登陆淘宝实例代码利用selenium 3.7和python3添加cookie模拟登陆的实现selenium自动化模拟键盘快捷键操作实现解析python走楼梯问题解决方法示例pandas pivot_table() 按日期分多列数据的方法python基于itchat模块实现微信防撤回利用scrapy将爬到的数据保存到mysql(防止重复)python 3.5下xadmin的使用及修复源码bugpython脚本执行cmd命令并返回结果的例子python实现的服务器示例小结【单进程、多进程、多线程、非阻塞式python生成九宫格图片python使用while循环花式打印乘法表一个基于flask的web应用诞生 用户注册功能开发(5)python入门教程 超详细1小时学会python 列表(list)操作方法详解python 元组(tuple)操作详解python 字典(dictionary)操作详解pycharm 使用心得(一)安装和首python strip()函数 介绍python 中文乱码问题深入分析python中使用xlrd、xlwt操作excepython科学计算环境推荐——anacpython逐行读取文件内容的三种方python 读写文件的操作代码python生成器的使用方法windows下pymongo下载及安装教程python 实现提取log文件中的关键句子,并进python实现连连看辅助(图像识别)python 生成不重复的随机数的代码python读写文件基础知识点python 网络编程常用代码段python执行子进程实现进程间通信的方法在pandas中处理nan值的方法
    免责声明 - 关于我们 - 联系我们 - 广告联系 - 友情链接 - 帮助中心 - 频道导航
    Copyright © 2017 www.zgxue.com All Rights Reserved