History on YouTube

Watch history on YouTube

Scrapy YouTube watch history spider. Because YouTube doesn't have a history search.

readme.md

Youtube History Scraper

A command line scraper written in Python 2.7 using scrapy and lxml. Scrapes your YouTube History into a searchable SQLite database, so you can search for something you've previously watched. Allowing search by video length, date published, video description, video title, etc. Uses pre logged in session cookies extracted from chrome.

This is just an experimental project. The database design isn't well thought out and there are not implemented methods for getting information back out of the database.

Ensure dependencies are met:

pip install scrapy lxml sqlalchemy

You'll need a session cookie from your web broweser, so the scraper can use your account to get your history.

Getting session cookies:

  • Click the "editThisCookie" button, then click export. The cookies are now on you clipboard.
  • Create the file "youtube_cookies.json" in this directory, then paste and save.

Alternitive (cookies) method:

  • Open chrome / chromium
  • Make sure your logged into youtube.com
  • Open the inspector (F12)
  • Open the network tab, check the "Preserve History" box
  • Visit
  • Click the request for "/feed/history" in the network tab
  • Copy the section "Request Headers", below the title "Request Headers"
  • Paste into a new file called "youtube_request_headers.txt" in this directory
  • In the settings.py file comment the "COOKIES_JSON" line and uncomment the "CHROME_HEADERS_FILE" line.

Run Scrapy:

scrapy crawl yth_spider

Profit?

Or use ipython and sqlalchemy, this example outputs all videos > 7 minutes in length:

from db_api import * db = AppDatabase with db._session_scope as s: h_entries = s.query(HistoryEntry).filter(HistoryEntry.time > 7 * 60).limit(5) for he in h_entries.all: print "{t} ( ".format(t=he.title, v=he.vid)

The db_api module is just a wrapper around a sqlalchemy model and session. db._session_scope gives you a sqlalchemy session you can run queries on, as shown above.

Source: github.com
RELATED VIDEO
How to Delete Clear YouTube View History from iPhone iPod iPad
How to Delete Clear YouTube View History from iPhone iPod iPad
How to Delete YouTube Watch and Search History on Lg G2
How to Delete YouTube Watch and Search History on Lg G2
How to Delete Watch History on YouTube
How to Delete Watch History on YouTube
RELATED FACTS
Share this Post

Related posts

Nixon Watch history

Nixon Watch history

SEPTEMBER 20, 2018

InChad DiNenna of Nixon Watches . Haadout photo. Steve Chapple INTELLECTUAL CAPITAL “We like to say a watch is halfway between…

Read More
How to Watch History Channel?

How to Watch History Channel?

SEPTEMBER 20, 2018

Watch hours of Vikings, Pawn Stars, Ancient Aliens, and more. All of your favorite HISTORY shows are available wherever and…

Read More