앱/봇 (1) 썸네일형 리스트형 How to Bypass Rate Limit for Web Scraping 1. Set User-Agent Using fake-useragent pip install fake-useragent from fake_useragent import UserAgent session = requests.Session() headers = { 'User-Agent': UserAgent().random } response = session.get(url, headers=headers) response.raise_for_status() 2. Use tor network for proxy Using tor: prerequisite: install tor browser pip install pysocks # tor browser를 키고 python 실행 url = "https://www.herme.. 이전 1 다음