Index with Url list - Problems

Post Reply
ddecook
Posts: 4
Joined: Wed Feb 27, 2002 10:25 am

Index with Url list - Problems

Post by ddecook »

I cron a group of shell scripts to index many sites into one database. I am having problems with one. Here is the output from the problem child:

Wiping todo list
Visited 0 pages total
Option: "-d/myindex_test"
Option: "-q8:00"
Option: "-r"
Option: "-noindex"
Option: "-meta=description"
Option: "-p15"
Option: "-v9"
Option: "-jhttp://www.mysite1.com"
175 Table hosts not found in data dictionary
115 No such table: hosts in the database: "-d/myindex_test"
000 SQLExecute() failed with -1
Option: "-jhttp://www.mysite2.com"
Option: "-xhttp://www.mysite1.com/browse.htm"
Option: "-xhttp://www.mysite1.com/guruslatest.htm"
Option: "-xhttp://www.mysite1.com/survey.htm"
Option: "-xhttp://www.mysite1.com/toolgurujuly.htm"
Option: "-xhttp://www.mysite1.com/askguru.htm"
Loading from lastrun
Using meta data field
: Disallowed protocol
Saving options and URLs to lastrun
Visited 0 pages
Visited 0 pages total
Host: ##IP##:80 (walkable) www.mysite1.com
restrict to:
/
Host: ##IP##:80 (walkable) www.mysite2.com
restrict to:
/
exclude:
/browse.htm /guruslatest.htm /survey.htm /toolgurujuly.htm /askguru.htm
getip() called 4 times. 0 hits
gethostbyname() called 4 times

The -p option is just for testing. When I run this script from the command line the out put is:
/home/web/https-webinator/webinator/bin/update_rw_1.sh: &Url1.lst: not found.

The Url1 file is:
-rw-rw-rw- 1 webin webin 4728 Jun 19 11:11 Url1.lst.

I use other list for indexing and have not had problems, not sure what has changed but I can't puzzle this one out...

Thanks dd
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Index with Url list - Problems

Post by mark »

Um, what's your gw command line? My guess a shell related syntax/usage error.
ddecook
Posts: 4
Joined: Wed Feb 27, 2002 10:25 am

Index with Url list - Problems

Post by ddecook »

User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Index with Url list - Problems

Post by mark »

Except it didn't translate well here. Basically the &Url1.lst is being treated as a separate command by the shell meaning you've broken it away from the gw command line somehow. Double check your script coding. This is not a webinator problem but a shell scripting problem.

BTW, your -j options are pointless. gw always stays on the specified hosts. -j is for staying under a subdirectory such as -jhttp://mysite/mydirectory/ .
Post Reply