| Author |
 Topic  |
|
|
onlinetimc
Starting Member
21 Posts |
 Posted - 24 Nov 2006 : 15:22:50
|
Hi Alessio,
Here is a suggestion I have which might make importing movies in bulk from the same web site a lot easier. (It kind of links in with the issues I was having before).
http://www.binaryworks.it/forum/topic.asp?TOPIC_ID=2342
What I would like to do is:
1. Use the CSV import to get the titles and other information I have for the movie already.
2. Use the batch import step 1 to automatically grab all the direct links (using option: Take First DirectLink Automatically). I have 400+ to add this once, this option hopefully will give me about 80-90% accurate links.
3. Then either:
a) Run a limited script which gets title and year only.
b) Just select title and year only to be updated
Then I can easily check what direct links have been selected without going to the website etc.
4. Run step 1 again on the ones I think are incorrect (clearing option: Take First DirectLink Automatically).
5. Finally, re-run the script to gather the full information.
In order for this to work, I would need to be able to either:
a) swap the "Script Used" field with the new full script
- no easy way to do this
b) trick the software into updating all the fields again (it tends to keep using whatever was last set when that script was run).
- can do this (I think) by running step1 & step2 with a single movie with all the fields selected again.
Before I try this, any thoughts or advice on an eaiser way to do this?
Also, when importing from CSV the length and year are filled in with "0". This stops being able to use the option "When Importing, Add New Values Only to Blank Fields..." to protect whatever I import via CSV. Is there anyway to stop this???
Thanks,
Tim
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 24 Nov 2006 : 16:09:20
|
I've just realised that if I am using the IMDB (complete) script it will also download covers etc and be very slow even if the title and year are the only fields selected. Therefore, the quickest option would be to use the IMDB (fastest) script during step 3 and then find a way to swap the "Script Used" for all the movies! Any ideas????
Tim. |
 |
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 24 Nov 2006 : 16:10:12
|
Hi Tim,
About CSV import, I will try to fix the problem about year and length.
About all others things, your suggestion is cool, but "automatize" all this process is not really easy to do.
I must think deeply to this, maybe with a new "tool" or advanced features, with precise steps to follow by user.
Thank you!
Alessio |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 25 Nov 2006 : 00:49:06
|
Perhaps a quick fix to save swapping the script used, is to simply rename the script! I didn't think of that until now.
Will let you know how the whole process goes, I think only the field selection is probably the only difficult bit (I know that'll be sorted soon enough). Only doing it the once so shouldn't be a problem.
Might be a useful extension to the import selection to scan the direct links (automated batch import step 1) and perform a "pre-script" or limited fields for the import. Of course it would be highly dependent on what the prefered source site is and not used by many (just those with old lists).
Many thanks.
|
Edited by - onlinetimc on 25 Nov 2006 00:52:05 |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 25 Nov 2006 : 13:16:39
|
Hi,
This morning I was thinking to this and perhaps I have found a way to do this (of course I must modify the engine):
1) Users, whenever possible, should write the year when adding a new movie
2) When you select directlinks, engine will compare the year of the movie with the year of the search-result webpage.
3) If there is a movie in this webpage that has the same name (or similar) and the same year, the engine will take the directlink automatically.
I think this is a good way for speed-up the directlink selection 
Alessio |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 25 Nov 2006 : 20:10:48
|
Hi!
I have complete the engine just now!
Seem to work correctly, I have make some tests and the engine is able to select directlink of almost all movies.
What you have to do is only fill the year in the movie before import it.
Alessio |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 26 Nov 2006 : 00:45:54
|
Sitting here shocked! You don't half work fast!
Many thanks, looking forward to trying it out. |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 26 Nov 2006 : 08:48:56
|
I was really excited about your idea, so I have work all yesterday on this. I hope to release the 5.5.7.0 tomorrow so we can start to deeply test it in order to clear bugs and maybe extend it to all scripts.
Alessio
From the news:
Preview of new "Intelligent Direclink Selection Engine"
From next release XMM will be able to automatically detect directlinks of many movies, what you have to do is only fill the movie "year". With this info, XMM will compare the movie year with the year on the webpage, and will select automatically the directlink without you do nothng. From next release this will be available on both IMDB scripts, but can be extended to all webistes that show year when you search for movies.
We are always a step beyond the others...
|
|
|
|
Mawu
Senior Member
Germany
1025 Posts |
Posted - 27 Nov 2006 : 12:48:43
|
Hello Alessio,
this sounds very promising.
Another additional suggestion: search by EAN-Code!!! For many dvd related sites (like DVD-Palace) a direct search by EAN-Code is possible and movie title + year is not always unique for different dvd releases.
bye
Mawu
|
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 27 Nov 2006 : 14:57:54
|
Hi Mawu,
You can add the EAN (called UPC) also now in the websearch:
#WEBQUERY#=...#UPC#
Maybe we can put some code in the #RIGHTPAGEFOUND# block.
For this, in the new 5.5.7.0 you can write this:
#GETMOVIEVAR#=#23#=#UPCVALUE#
to read the UPC of imported movie.
Alessio |
|
|
|
Mawu
Senior Member
Germany
1025 Posts |
Posted - 27 Nov 2006 : 17:33:08
|
@Alessio
Fine, and how does it work during the import procedure?
My suggestion how it should work:
1.Add a new movie manually by EAN
2.Type the EAN into a pop-up field
3. Press button Quick import with default script. Script is configured to search by EAN and not by title.
The whole procedure should be as easy as possible.
bye bye
mawu
|
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 27 Nov 2006 : 18:17:34
|
We can do this:
If a movie have the EAN code, the script try to search directly for this, otherwise the script search the title of the movie.
I need some time to write the code and test it.
About the 1), do you mean a window where user put only the EAN (I mean a window with only this textbox?)
Bye!
Alessio |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 27 Nov 2006 : 22:01:43
|
Hi Alessio,
Been running some tests on the latest build (B7). I ran a bulk import from CSV and then search for links. After checking the titles the automatic import got about 15% wrong. Don't worry that wasn't using the new import method (I didn't have the year info availble unfortunately).
1. The year and length is still 0 when not imported.
2. There is now no way to do manual selection of the results from IMDB, even with the automatic selections "Take First DirectLink Automatically" and "Use Inteligent Directlink Engine" disabled and clear directlinks enabled. This appears to be the case now with newly added movies too. I think this has broken with the new directlink engine.
2 is a bit of a problem since I can't select the correct links for the 15% which weren't correct from the automatic selection. Anyway, I will have another try today and see what I can get going.
Many thanks.
Tim.
-----------------------------------------------
UPDATE:
Many thanks to Oliver678 updating the IMDB scripts, the manual selection appears to be working now (if it was in fact broken at all). Thanks for the hard work from you both.
|
Edited by - onlinetimc on 28 Nov 2006 12:32:52 |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 28 Nov 2006 : 12:34:07
|
Hi Tim,
It's strange, I haven't such problem with new IMDB script. I have try just now the movie "Rambo", I have set the movie year to 0.
XMM correctly doesn't find a match and show me directlinks to choose.
Please let me know,
Alessio |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 28 Nov 2006 : 13:39:27
|
Alessio,
Well the new IMDB scripts added this morning appeared to fix this anyway (I don't know what had changed in them). It might just have been a case of restarting the computer fixed it somehow. Anyway, that is working fantasticly at the moment (in full swing with my big import).
The year and length problem is from importing via CSV file. It doesn't cause any issue for me since none of the data I import overlaps with what the scripts import so I don't need to protect the other fields.
Will post up some (hopefully) useful suggestions once I've finished my import. Thank you for all the help in getting this far!
-----------
EDIT:
Got through 16 with being able to select manually and then it started adding the links automatically (1 Direct Link found) for the remaining 41 movies (even though I cleared the links in the same way for all)! I will investigate this a little more and let you know what the issue is.
(This is doing a batch import using Step1 to select links first.)
Things so far: It doesn't matter what the title is or if its a newly added one, any film will be updated with 1 Direct Link found. After it starts to do it, the next/1st title is set to "IMdb Tile Search". Restart of the software, allows manual selection again! Very strange!
Settings are:
IMDB (Fastest)
All directlink tick boxes unselected
Title Import: True
Title Year: Ture
Import Fields: Title, Original Tile and Year.
Selected Clear all OLD links.
Just found that the movie title which causes it in this case is:
M.I.B. Men In Black (with the year set to 1997, since it previously incorrectly linked with "Men in Black: The Series" (but I don't think that is the problem, I expect its the M.I.B. part).
Hope that helps get to the bottom of it!
EDIT:
I.Q. doesn't cause it, but Anita & Me does but only when I "Searched Again" with Anita And Me! Anyway, not a big problem since I only have 8 movie links left to sort out.
Once I've finished my imports I'll see if they are repeatable and what does and doesn't work.
Update:
It appears to happen in cases where an IMDB search would jump directly to the result (i.e. only one result found). The problem is it automatically selects this link and then takes the first link for all the rest. Also the title gets set to IMDb Title Search and any other imports until the program is restarted is wrong. |
Edited by - onlinetimc on 28 Nov 2006 23:45:47 |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 29 Nov 2006 : 14:00:03
|
Just a quick question Alessio,
When you run a "Quick with Default Script" update, what does the software do if the item already has the correct direct link etc.
Does it:
1) Search for a new link and select the first it finds
2) Uses the direct link which is already there (even if its not what IMdb would find from the title)?
Just don't want to loose any links I've manged to find (don't want to change the titles particularly for foreign titles).
Many thanks (almost there with my importing!!!!).
Tim.
UPDATE:
Just tested this by cloning an imported movie (The Grinch (2000) - imported with IMDB complete but only with title and year) and renaming the title (to TestMovie, and year to 2020) and running a quick update:
- It searches again for the movie...:o(
- If I skip it, it imports the data anyway from the oringal directlink but only with the previously selected fields from the original import.
- If I then run Quick update again (this time it has year 200), it automatically selects the first link it finds (AFI's 100 Years..) and imports all! (which I think links to the previous problem when there is only one link found).
The only way to get it to re-use the direct link already found is to run a (Step2) Batch import, but the fields which get updated are set to whatever was used previously (for that movie) and there is no way to reselect them.
How can I get the batch import to import all fields? Or how can I stop the software from trying to search for the link when there is already one there?
Sorry, I'm hoping I've just missed a setting somewhere which will do this but I've tried a few things and can't get it to do what I need whatever I try! |
Edited by - onlinetimc on 29 Nov 2006 16:09:10 |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 29 Nov 2006 : 14:57:43
|
Hi,
Please sorry I answer so late. I must leave now, I will read your post better in few hours.
Alessio |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 29 Nov 2006 : 16:45:46
|
Many thanks. Let me know if it doesn't make any sense and I'll try to be a little clearer about the problems.
Thanks.
---------
UPDATE: Sorry about posting a lot, helps me think aloud ways around the issues. I think I can manage by searching using the original titles for now (I'll add a bug report too, which will hopefully summarise the other issues better).
Here's my suggestion on how the above idea(s) could be done (without hopefully too much trouble and be easily used for all scripts).
I think half the problem is how I started doing the importing (which became a bit of a mess), where I was trying to get the best of both worlds.
- 1st making the use of the automatic selection of links to get the bulk of the links (332/389 were clearly correct, the rest were either slightly out or I wanted to check).
- 2nd I made use of the manual search to select the rest of the links. The real problem I had was I wanted to download all the information AFTER I was sure the links were right.
Perhaps with a slight adjustment to the steps it would be possible to do something similar...
1. Run step one as before...take first DirectLink, skip automatically if there are no Directlinks, full import, all fields.
2. When it comes to just before the Next Phase (i.e. with quick import off). You can include the title of link (which would be title and year etc etc) and perhaps even the weblink for each movie in the information box, these can then be reviewed by the user before proceeding. You could then have the option to set to skip a movie and then allow the user to:
a) re-run a manual search on the ones skipped or...
b) continue with the import but skipping the movies where required.
This would mirror what I was (in my twisted attempt) trying to do.
|
Edited by - onlinetimc on 30 Nov 2006 14:17:35 |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 02 Dec 2006 : 12:18:46
|
Hi onlinetimc,
Finally I have found some time to asnwer you.
By reading your post I think that a solution could be to let users "edit" the fields to re-import with batch-import.
So, the goal is to select directlinks ONLY ONE TIME, and then have the possibility to edit, for every script used, the fields to update when you do the batch update again and again.
What do you think? this will be really not easy to code, but I will try.
Alessio |
|
|
|
onlinetimc
Starting Member
21 Posts |
Posted - 04 Dec 2006 : 11:46:34
|
Hi there,
Thank you for your time and effort with this.
I think there are two issues here.
1. The field selection for the script. Yes I think some way to edit the fields for the script will be ideal (I don't know how this information is kept but it appears to be part of the movie record itself, so I can understand that its quite a complicated problem).
At the moment, its probably not a top issue and creating a way to clear individual fields and only replacing them if empty would cover most situations anyway. For now, would it be any simpler to only allow selection of the fields in the first step and then allow just this selection to be cleared to default if required. Again, if the directlink can be kept on import, a re-selection of the field can be done in step 1 anyway and no other changes for this would be needed.
2. If you allow the direct links to be reviewed before Step 2, then it feels like a more logical process than a part import and re-import. This would avoid the need to swap scripts, change fields, protect directlinks etc etc.
The GOAL for me was to avoid importing lots of incorrect information by getting the links (webpage link to the movie) automatically and then reviewing them before performing the import. So for the way I did it, it was important to keep the links once I'd reviewed them. However, for most cases I'd imagine you'd want to keep the links for each website you import from. I don't know how they are managed at the moment, but guess a system of keeping several links to different sites which relate to each script i.e. stored IMDB link which can then be refered to by any IMDB script and stored Amazon link which can be used by any Amazon script. That way the user would be able to build up a list of related links to the movie and be sure that the right information will be imported. Of course, I fully understand things may be different and changes although easy to speculate on represent a huge amount of time and effort to implement (if they even fit in with what currently is there).
Anyway, I have imported all the movies from my list now and very happy with the results. Sorry for being a difficult user by trying things differently but thank you for your help. Please don't worry too much since I am not waiting on any changes, my importing is done and happy using the software. |
Edited by - onlinetimc on 04 Dec 2006 11:49:55 |
|
|
|
Alessio Viti
Forum Admin
Italy
9171 Posts |
Posted - 06 Dec 2006 : 18:15:53
|
Hi onlinetimc,
Sorry if I answer always late to your posts, but I must read it carefully 
My basic idea is to make another "tab" in th Edit window, where there are the Script you have used, the directlinks selected, and the fields checked.
In this new tabs, userscan quickly re-select the field to import for every script, without use the "step 1" again.
I am sorry, my english is bad and for me is hard explain correctly what I think.
Thank you!
Alessio |
|
|
|
Topic |
|
|
Easy automatic importing....