How to avoid HTTP error 429 (Too many requests) in python while using pytube library

Original article can be found here (source): Deep Learning on Medium

How to avoid HTTP error 429 (Too many requests) in python while using pytube library

I have been using pytube library to do some test with youtube videos.

But within a single day; I started to get HTTP 429(too many request) errors.

Which is fair; I should have placed more sleeps between download requests.

The problem is; although I have updated my code to place more sleeps; after 3 days I was still banned. Which I think is not fair because I made a mistake; I did not intentionally tried to download all of YouTube 🙂

Anyways;

interestingly I was able to use YouTube from browser; so it is not an IP level ban. There should be some piece of code on YouTube side which checks one of the request headers and let me download.

Unfortunately pytube library do not accept request headers as a parameter. So I had to clone it(here) and create a new class called YouTubeItem.

And then I was able to pass request headers.

I first tried to pass the user agent;

youtube_item = YouTubeItem('https://www.youtube.com/embed/B48hwisZvEI', request_headers={
'User-Agent':
'<my user agent>',
})

but it did not work;

then I tried to pass the youtube client name and youtube identity token.

youtube_item = YouTubeItem('https://www.youtube.com/embed/B48hwisZvEI', request_headers={
'User-Agent':
'<my user agent>',
'x-youtube-client-name':
<my client name>,
'x-youtube-identity-token':
'<my identity token>',
})

AND…

it did not work:)

then I tried to add cookies;

youtube_item = YouTubeItem('https://www.youtube.com/embed/B48hwisZvEI', request_headers={
'User-Agent':
'<my user agent>',
'x-youtube-client-name':
<my client name>,
'x-youtube-identity-token':
'<my identity token>',
'cookie': '<my cookie>'
})

AND…

it did work!

Amazing right?

Ok one last thing; where did I get these request parameter values? Just open youtube on your browser; and open developer tools. Then open the Network tab. and then select a youtube video. You will see an outgoing request on the network tab. There you can find your headers.

BUT BE CAREFUL;

if you mis-use the library AGAIN; you may get your IP banned or worse; you can lose your google/youtube account.

So only use this workaround if you are sure you took the necessary measures to not misuse it again.

Thanks for reading!