Little explanation on 'Depth'

Post Reply
b.sims
Posts: 99
Joined: Fri Oct 26, 2001 10:40 am

Little explanation on 'Depth'

Post by b.sims »

Can I use the 'Depth' parameter to limit the crawl to a subdirectory of a site?

I want it to crawl www.mysite.com/asubdirectory so that all links on pages in that directory will be checked esp. for broken links. However, I don't want it to read pages in www.mysite.com/anothersubdirectory. At the moment it seems to be limiting itself to mysite.com but is escaping to other sub-directories.
User avatar
John
Site Admin
Posts: 2597
Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:

Little explanation on 'Depth'

Post by John »

John Turnbull
Thunderstone Software
b.sims
Posts: 99
Joined: Fri Oct 26, 2001 10:40 am

Little explanation on 'Depth'

Post by b.sims »

Thanks. However, doesn't this cause all pages without the specified prefix url to be ignored? If I have a link to www.aseperatebrokensite.com this link will be ignored and therefore I will not be told it is broken, right?
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Little explanation on 'Depth'

Post by mark »

Not quite. The directory restriction only applies to that site. gw won't try offsite urls anyhow unless you use -o. Maybe you want
gw -d- -D1 -o http://www.mysite.com/asubdirectory/
Post Reply