XBMC Community Forum  

Go Back   XBMC Community Forum > Development > Scraper Development

Scraper Development Developers forum for meta data scrapers. Scraper developers only!
Not for posting feature requests, bugs, or end-user support requests!

Reply
 
Thread Tools Search this Thread Display Modes
Old 2009-10-31, 18:05   #1
KoTiX
Fan
 
Join Date: Jun 2004
Posts: 376
KoTiX is on a distinguished road
Default [RELEASE] Movieplayer.it scraper for Movies new version (italian)

With permission from Sipontino I updated his movies scraper, it download everything it can find on the movieplayer.it site, including:

- posters
- fanarts from wallpapers, promotionals and scene pictures
- actors thumbs
- plot
- premiered
- rating and votes
- trailer
- director and writer
- genre
- year

also form other sites:

- TMDB poster and fanart
- MPAA, studio and ratings from IMDb

UPDATES:
20/11/2009 - Added option to retreive up to 4 pages of results from mp.it it' will take longer obviously.
17/11/2009 - Added staright MP search (tnx Cimi for the great help), still possible to use Google search enabling it in the scraper settings.
16/11/2009 - cleanup of results title names for future XBMC filtering of duplicated entries (r24674 )
15/11/2009 - fixed "outline" problem with the Rapier skin, it actually contain the plot and not the curiosity anymore.
14/11/2009 - changed search url with myown google custom search to have more results
12/11/2009 - changed method to retrieve the IMDb ID and the so the MPAA, studio, rating and Fanart
09/11/2009 - added scraping of MPAA, studio, and ratings from IMDb (enabled by default, if disabled it will retrieve thos from movieplayer.it but the datas will probably be unusable by skins that use MPAA and studo icons)
04/11/2009 - added download of TMDB Posters and Fanarts and fixed their selection in settings

notice that TMDB Posters will be the last in the list and they will be in english, for Fanarts you'll have to choose between TMDB or movieplayer source as they cannot be added sequentially like for Posters (xbmc limitation), so I suggest you to use movieplayer for italian movies and TMDB for all the rest.

Enjoy.

Download the Final Version

Last edited by KoTiX; 2009-11-20 at 21:42.
KoTiX is offline   Reply With Quote
Old 2009-10-31, 19:17   #2
sipontino
Junior Member
 
Join Date: Dec 2008
Posts: 23
sipontino is on a distinguished road
Default

Great!!
work like a charm.
Now ill go on test on it!!

Last edited by sipontino; 2009-10-31 at 19:19.
sipontino is offline   Reply With Quote
Old 2009-11-02, 10:05   #3
chareos
Member
 
Join Date: Jun 2009
Posts: 57
chareos is on a distinguished road
Default

This is really nice !
Only thing off, this resource provides odd-low resolution, 4:3 fanarts (which look so bad in xbmc)

Is there a way to get data from movieplayer.it AND cover-art and fan-art from other resources, like imdb ?
chareos is offline   Reply With Quote
Old 2009-11-02, 13:49   #4
KoTiX
Fan
 
Join Date: Jun 2004
Posts: 376
KoTiX is on a distinguished road
Default

Here you go a new version that download Posters and Fanarts from TMDB (check first post).
I'm still working on it so for now you need to disable Fanarts from movieplayer.it in the settings and enable the TMDB one to get it to work properly.
Enjoy
KoTiX is offline   Reply With Quote
Old 2009-11-03, 02:10   #5
KoTiX
Fan
 
Join Date: Jun 2004
Posts: 376
KoTiX is on a distinguished road
Default

Ok I got them fixed now
KoTiX is offline   Reply With Quote
Old 2009-11-03, 12:36   #6
chareos
Member
 
Join Date: Jun 2009
Posts: 57
chareos is on a distinguished road
Default

This is great news ! I'm going to try later today !

ps: is this going for your tv-series scraper too ?
chareos is offline   Reply With Quote
Old 2009-11-03, 22:42   #7
chareos
Member
 
Join Date: Jun 2009
Posts: 57
chareos is on a distinguished road
Default

Tried it !

Worked GREAT with "Apocalypto" (italian and original title are the same), italian informations, hi-quality arts from TMDB...
you're getting there, man !

Did not work as intended for "Angeli e Demoni" (original title is Angels & Demons)

I did a few tests, my guess is
- if a file has the italian title, fanart won't be find on TMDB
- if a file has the original title, it won't be found on movieplayer.it

The solution I'd suggest (if technically possible) is:
1 - search movieplayer.it for the file title
2 - (get data, art, etc)
3 - get the original title value (try search for "Angeli e Demoni": on the right, the original title is provided).
4 - get the year
5 - search TMDB for the original title and year found in step 3+4 (to reduce false positives)

This should be all the MAGIC we may ever need for the ultimate italian scraper.
What do you think ?
chareos is offline   Reply With Quote
Old 2009-11-04, 01:25   #8
KoTiX
Fan
 
Join Date: Jun 2004
Posts: 376
KoTiX is on a distinguished road
Default

It already search for the original title, the year is not used yet, anyway the problem with Angels & Demons is the & that was missunderstanded by google, i got it fixed now using urlencode="1" :P
The initial search is also improved now having just one result per search.
Link in first post is updated.
KoTiX is offline   Reply With Quote
Old 2009-11-04, 02:10   #9
KoTiX
Fan
 
Join Date: Jun 2004
Posts: 376
KoTiX is on a distinguished road
Default

Ok I got the search for Original Title + Year working and this time it search directly on Imdb.com (we don't need google anymore to look into Imdb).
We need to find out which site is better for our (italian) searches between Imdb.com and Imdb.it
KoTiX is offline   Reply With Quote
Old 2009-11-04, 10:36   #10
chareos
Member
 
Join Date: Jun 2009
Posts: 57
chareos is on a distinguished road
Default

I'll try it in a few minutes !


BTW ...
Quote:
notice that TMDB Posters will be the last in the list and they will be in english, for Fanarts you'll have to choose between TMDB or movieplayer source as they cannot be added sequentially like for Posters (xbmc limitation), so I suggest you to use movieplayer for italian movies and TMDB for all the rest.
1 - Would you consider to write to XBMC developers to ask for a solution to that limitation ?
2 - Is it technically possible to add a prefix (HQ) to the art fetched from TMDB (high quality) in order to help newbies to realize how the scraper works ?

I think those two points could move the scraper forward to an official merge with xbmc itself... which would be great
chareos is offline   Reply With Quote
Reply

Bookmarks


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 21:22.


Protected by Akismet, We recommend WordPress blogs
Copyright © 2008, XBMC Project