YouTube
YouTube reco system
1. Considers personal activity i.e. favorites, liked, videos watched, people subscribed to, videos watched till the end. 2. Build a co-visitation graph of expanding set of videos.
Issues to consider for YouTube
1. How to rank 2. What info is available about a video 3. Reco system 4. Distribution system 5. How to monetize the website.
YouTube info
1. In Dec 2016, ranked as 2nd most popular site by Alexa internet 2. Hulu, CBC also upload and share some of their content as a part of YouTube partnership program.
YouTube ranking factors
1. Metadata 2. HD > lower quality 3. # of views likes and shares.
Ranking in video search engines
1. Metadata and user prefs. 2. Upload date. 3. Duration. 4. User ratings. 5. Views.
What info YouTube collects about a video
1. Name 2. Description 3. Title 4. Tags 5. Thumbnails 6. Copyright information 7. Age restrictions.
Youtube as a content aggregator
1. Not a search engine in conventional sense 2. Only searches videos uploaded by users. 3. Doesn't return anything except for videos in search results. 4. Does not crawl the web for content.
YouTube upload flow
1. User clicks upload from desktop. 2. Video sent to San Jose data center 3. Video is uploaded to Server Breach located in Louisiana / Virginia 4. transcoded video copies sent to LA, Chicago, Ashburn, Washington DC
Details on ContentID
1. User uploads video 2. Transcoded to different formats 3. Spectrogram used for hashing audio 4. Sample sections of video are hashed 5. Matched against audio and video hashes of content submitted by copyright owners.
What does metadata info include for ranking by YouTube
1. Video titles 2. tags 3. descriptions. 4. Links to websites and social profiles.
Association Rule Mining
1. c(i, j) represents the co-visitation count for videos vi and vj 2. The relatedness of these two videos is defined by c(i,j) / f(i, j) 3. f(i, j) = c(i) * c(j) is a normalization funciton that takes into consideration the global popularity of individual videos.
Caveats anf Facts about YouTube
1.8 different video formats supported 2. 128Gig max video size 3. Default allowed duration 15 minutes but can be extended 4. Videos are never delted by YouTube.
YouTube creation
3 former PayPal employees in Feb 2005
YouTube's caching architecture
3-Tier Hierarchical Caching architecture. 38 primary caching locs 8 secondary locs 5 tertiary locs, spread over 5 contintents.
What is the % of video clicks accounted for by recommendation videos
60%
YouTube identification method for videos
64 bit, 11 character fixed length unique string per video.
What % video rich snippets are YouTube's
91%
Content Delivery Networks
A collection of content servers with the intelligence of being able to select a subset of servers based on a user's location and the content being requested.
Video search engine.
A search engine that crawls the web for video content.
CDN service providers
Akamai Limelight Level 3
What devices can play YouTube videos
Android devices Apple phones XBox, Playstation Apple TV, iPod touch
Why does the reco system need to be updated frequently?
Because enw videos are added very frequently and a video may go viral anytime and hence alter the statistics greatly.
Some video search engines
Bing Munax Science Stage
YouTube Monetization Copyright issue resolution
Copyright material identification system built by Google. Allows the original owner to either take down the content or let the content remain and allow ads to appear and the revenue is split two ways between YouTube and the owner.
What is ContentID
Database created by YouTube for keeping fingerprints of copyrighted content.
Indexing in video search engines
Done by acquiring metadata related to the video: 1. Quality 2. Author 3. Title 4. Creation date. 5. Tags 6. Description.
When does YouTube collect info about a video
During upload time
Vimeo
First to allow HD videos, focuses on short artsy videos.
Co-visitation relatedness graph
For a given seed video, we find top N ranked videos by the relatedness formula in association rule mining. Related videos form a directed graph.
What special thing had to be done to play YouTube videos on Apple devices
Had to be encoded in H2.264 which is Apple's preferred video format
Video streaming services
Hulu, Netflix, Amazon Prime.
Vevo
Joint venture b/w Universal group, Warner music group and Sony entertainment
Disecting rare videos study
Made a call for a rare video in Cali. It was initially served from Netherlands but eventual calls were served from Cali. So, videos are moved to where they are wanted.
Do all video search engines crawl the web for content?
No, some rely on videos being uploaded by users e.g. YouTube.
YouTube Search Engine stats
Processes more than 3 billion searches a moth which is more than Yahoo, AOL Ask combined. 4 billion videos a day 800 million unique users a month 70% traffic outside of US.
What does YouTube want to achieve with their CDN
Reduce RTT (round trip time)
What is spectrogram
Time frequency graph for audio files. Peak points are marked as a part of the hashing process for ContentID copyright matching.
Video hosting services
Vimeo Vevo Dailymotion YouTube
Video rich snippet
When someone searches for a query on google, a small video also pops up telling the user there is a video to help on the topic. Google is accused of being biased towards displaying YouTube videor rich snippets.
Is it legal to crawl YouTube?
Yes
In addition to the normal stuff for indexing in video search engines, what else is used
transcription and subtitles.
