| T O P I C R E V I E W |
| Prinz |
Posted - 19 Mar 2011 : 18:45:42
I tried to import with the IMDB.COM (TVSeries) For Rel.7 the TV Show: "The Young and the Restless" but the Awards don't get imported because of a Magic Script Bug.
If i try it with the Magic Script Editor it jumps directly after trying to load the Awards Page (http://akas.imdb.com/title/tt0069658/awards) in Script Line 2703:
#OPENNEWWEB#=#26#
into to #CUSTOM3# Block  Maybe the Award Page for this long-running Show is to big (to many Lines).
And there is another unrelated Problem with Shows with that many Episodes: The Episode Chooser Dialog takes practically forever to fill with all the Episodes, in this Time XMM is practically unusable and must be terminated via the Task-manager if you don't want to wait until the next Day for the List to fill...  |
| 20 L A T E S T R E P L I E S (Newest First) |
| JDommi |
Posted - 08 Apr 2011 : 08:34:05
Ah, Thx Prinz!
And here is the full list for the search sections (parameter 3) last time I used it:
quote:
' *****************************************
' Sektionen:
' *****************************************
' Sonstige
' "" ->Alle Kategorien
' "9" ->XXX
' Audio
' "2" ->Audio-CD
' "10" ->Audio-DVD
' "16" ->Audio-VHS
' "18" ->Kasette
' "19" ->Audio-SVCD
' "5" ->Audio-Maxi-CD
' "12" ->Hörspiele
' "13" ->Vinyl
' Filme
' "1" ->Video-SVCD
' "3" ->Video-DVD
' "11" ->Video-VHS
' "15" ->Video-DivX
' "17" ->Dokumentation
' "6" ->Serien
' "33" ->HD-DVD
' "32" ->Blue Ray
' Spiele
' "4" ->PC-Games
' "8" ->PS1
' "7" ->PS2
' "31" ->PS3
' "20" ->X-Box
' "29" ->X-Box 360
' "21" ->Gamecube
' "22" ->UMD
' "30" ->Wii
' Programme
' "14" ->PC Programme
' *****************************************
|
| JDommi |
Posted - 07 Apr 2011 : 18:13:59
I know that cover-paradies doesn't need the post command. The code is a really old example for "POST".
@Ale: there are needed two numbers. The exact values I will send to you tomorrow (I'm @work at the moment) |
| Prinz |
Posted - 07 Apr 2011 : 17:16:59
You don't need post data for that site. Just use the extended search:
Example:
http://cover-paradies.to/?Module=ExtendedSearch&SearchString=Battle+Los+Angeles&StringMode=All
The Post Data for the simple search url http://cover-paradies.to/?Module=SimpleSearch would be:
Page=0&SearchString=Battle+Los+Angeles&Sektion=&B1=Search%21 |
| Alessio Viti |
Posted - 07 Apr 2011 : 16:36:07
Thank you JDommi!
XMM is write in vb .net... I will make some test and let you know.
So, for cover paradies for example we need the url and the postdata, right? so I have to add a new variable to script to store these info.
Or maybe I can add just an option to enable the use of POST and use the
#USEPOST#=1
#SCRIPTWEBSITE#=http://www.cover-paradies.to/?Module=SimpleSearch
for the website, and then
#WEBQUERY#
for the post data
Sorry... do you know exactly what is the postdata string to use with cover paradies?
Ale
|
| JDommi |
Posted - 07 Apr 2011 : 10:01:30
Well, haven't done such thing in VB but isn't it that simple
Dim inputData As String = "[search-term]"
Dim postData As String = "param1=" + inputData
myHttpWebRequest.ContentType = "application/x-www-form-urlencoded"
Dim response As String = Encoding.UTF8.GetString(myHttpWebRequest.UploadData(url , "POST", Encoding.UTF8.GetBytes(postData)))
As WShell-Script I only have used this code, but it opens an IE window.
On Error Resume Next
strURL = "http://www.cover-paradies.to/?Module=SimpleSearch"
param2 = WScript.Arguments(0)
param3 = WScript.Arguments(1)
Set IE_Instanz = CreateObject("InternetExplorer.Application")
Do While IE_Instanz.Busy
Loop
IE_Instanz.Visible = 1
IE_Instanz.Navigate strURL
Do While IE_Instanz.ReadyState <> 4
Loop
Set IE_Doc = IE_Instanz.Document
IE_Doc.all.Page.value = "0"
IE_Doc.all.SearchString.value = param2
IE_Doc.all.Sektion.value = param3
IE_Doc.all.B1.Click
Set IE_Doc = Nothing
Set IE_Instanz = Nothing
in VBNet
Dim req As Net.HttpWebRequest
Dim resp As Net.HttpWebResponse
Dim str, str2 As IO.Stream
Dim cookies As Net.CookieContainer
Dim buffer As String
Dim data As Byte() = System.Text.Encoding.ASCII.GetBytes(*postdata*)
req = Net.WebRequest.Create(url)
req.CookieContainer = cookies
req.Method = "POST"
req.Accept = "*/*"
req.Headers.Add(Net.HttpRequestHeader.AcceptLanguage, "de")
req.Headers.Add("UA-CPU", "x86")
req.Headers.Add(Net.HttpRequestHeader.AcceptEncoding, "gzip, deflate")
'Fürn Post Request den ContentType & Lenght + Data
req.ContentType = "application/x-www-form-urlencoded"
req.ContentLength = data.Length
str2 = req.GetRequestStream()
str2.Write(data, 0, data.Length)
str2.Close()
resp = req.GetResponse
str = resp.GetResponseStream()
If (resp.ContentEncoding.ToLower().Contains("gzip")) Then
str = New IO.Compression.GZipStream(str, IO.Compression.CompressionMode.Decompress)
ElseIf (resp.ContentEncoding.ToLower().Contains("deflate")) Then
str = New IO.Compression.DeflateStream(str, IO.Compression.CompressionMode.Decompress)
End If
buffer = New IO.StreamReader(str, System.Text.Encoding.ASCII).ReadToEnd 'only for ASCII |
| Alessio Viti |
Posted - 07 Apr 2011 : 01:16:49
Yes, indeed a post command will be useful... but I don't know how to do it right now
Ale |
| JDommi |
Posted - 06 Apr 2011 : 23:20:07
Do we need that command anyway? I don't think so!
But maybe we could need such funtion some day...
What really would be a good thing: implementing a post command |
| Alessio Viti |
Posted - 06 Apr 2011 : 23:15:55
Sorry guys, my mistake.
the #READPREVIOUSLINE# doesn't exist!
I will create it. I was think to #FINDPREVIOUS#
Ale |
| Alessio Viti |
Posted - 06 Apr 2011 : 23:04:02
Wow Prinz,
with latest release of script the cycles of awards section (Inception) fall down to 38.000 instead of 62.000!
Good job with script!
Ale |
| Alessio Viti |
Posted - 06 Apr 2011 : 20:16:00
Thats strange, I use it in the XGM and XBM and seem to work. I will check.
Thank you! |
| Prinz |
Posted - 06 Apr 2011 : 15:31:12
quote:
Originally posted by JDommi
#READPREVIOUSLINE#
That function doesn't work at all. As long as i can remember it never worked. |
| JDommi |
Posted - 06 Apr 2011 : 09:20:02
To each saint his candle: it was Prinz' idea!
Just send a new magicscript.htm to Ale.
Accordingly to #READNEXTLINE# the command #READPREVIOUSLINE# should be modified, too. |
| Alessio Viti |
Posted - 06 Apr 2011 : 01:01:37
I have modify the #READNEXTLINE# as suggested by JDommi.
This is the result:
Inception:
Instructions executed with old/new #READNETLINE#:
74644/62819
With Matrix:
56664/49009
The Script can be improved by removing empty lines in script, or maybe by removing some instructions. I will take a look. |
| Alessio Viti |
Posted - 05 Apr 2011 : 01:11:23
That's a good suggestion JDommi!
I will try to add it.
Ale |
| Prinz |
Posted - 03 Apr 2011 : 16:46:04
I have a solution for that problem. Don't add a new function just add a parameter to the existing function #READNEXTLINE# like this:
#READNEXTLINE#=1
Old/current version will ignore the =1 and just go to the next line. The new XMM Version should go to the next non-empty line if =1 is added as a parameter. So there are no endless loops.
So you don't need to program and maintain 2 Scripts, in old/current XMM Versions the import would be simply slower as in future XMM Versions. |
| JDommi |
Posted - 03 Apr 2011 : 16:08:13
I would suggest to make a second script with a Version in filename like the V7 for TV Shows. |
| Prinz |
Posted - 03 Apr 2011 : 15:52:50
There is only one problem with this new function #READNEXTNONEMPTYLINE# .
If scripts change to this new function old XMM versions will end up in a endless loop with this functions, because old XMM Versions don't know that function and will loop on the same line forever... |
| Prinz |
Posted - 03 Apr 2011 : 02:38:48
It's really unbelievable how many empty lines are on the imdb.com awards page. 
Example "Lost" (TV Show) Awards Page:
Only the relevant lines: 9956 Lines
After removing empty lines with linux command:
grep -v "^\s*$" org.html > rem.html
4215 Lines
Which means 5.741 empty Lines and the File size was reduced from 205kB to 139kB.
That is also a huge waste of there server bandwidth. |
| Alessio Viti |
Posted - 03 Apr 2011 : 01:29:21
Hello Guys,
I will try to implement this new function, that skip the line if contains nothing or only spaces.
#READNEXTNONEMPTYLINE#
Thank you for the suggestion!
Ale |
| JDommi |
Posted - 02 Apr 2011 : 22:17:02
Okay, convinced! |
