Large indexes and live search

scott.shaver · Post by **scott.shaver** » Fri Mar 17, 2006 11:09 am

Okay I'm confused again

. I have been indexing one of our file servers the walk has been running for nearly 48 hours. right now it seems to only be updating the

pages scheduled to be refreshed

value.

0 pages in todo
317,652 pages scheduled to be refreshed
15,079 pages visited in the last hour (14,811 success/268 failed)
371,418 pages in index

What exactly is this doing? When I go to the live search and do searches for things I know it should be returning tons of results I only get a few like 1 or 2.

Do I have to wait for the above process to finish before I will see the correct search results? The total size of the data directory for this index is now at 5.34 Gig.

Post by **John** » Fri Mar 17, 2006 11:50 am

The 15,079 visited in the last hour seems to suggest it is still running. What does the bottom of the Walk Status show?

You can also do a "Pause and Live" to pause the walk, build the index on the crawled data, and make it the live walk. A refresh would then continue that walk.

scott.shaver · Post by **scott.shaver** » Fri Mar 17, 2006 12:01 pm

Okay so you think it has built the index yet. Here is the entire walk status page:

0 pages in todo
315,510 pages scheduled to be refreshed
12,507 pages visited in the last hour (12,242 success/265 failed)
371,435 pages in index

Pages recently walked
371435 pages (689,981,323 bytes) so far.
94371 errors so far.
0 duplicate pages so far.

Page Visited Modified Url
-------+-------------------+-------------------+-------------------------------------------------------
371435 In less than 1 min 648 d, 3 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2057257_EMC%20CORK%20IRELAND_MCTX/06.07.04_so%2057257_EMC%20CORK%20IRELAND_MCTX_.xls (187,904 bytes)
371434 Less than 1 min ago 651 d, 35 mins ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04_mo%2062185_DLIE%20HUB%2009Y556_MCTX/06.04.04_mo%2062185_DLIE%20HUB%2009Y556_MCTX_.doc (138,240 bytes)
371433 Less than 1 min ago 648 d, 3 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2057257_EMC%20CORK%20IRELAND_MCTX/06.07.04_so%2057257_EMC%20CORK%20IRELAND_MCTX_.doc (139,776 bytes)
371432 Less than 1 min ago 651 d, 1 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04_mo%2062181_DLIE%20HUB%200C0456_MCTX/06.04.04_mo%2062181_DLIE%20HUB%200C0456_MCTX_.xls (134,144 bytes)
371431 Less than 1 min ago 651 d, 3 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04_mo%2062092_DLIE%20HUB%200C0456_MCTX/06.04.04_mo%2062092_DLIE%20HUB%200C0456_MCTX_.xls (134,144 bytes)
371430 Less than 1 min ago 651 d, 1 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04_mo%2062181_DLIE%20HUB%200C0456_MCTX/06.04.04_mo%2062181_DLIE%20HUB%200C0456_MCTX_.doc (140,800 bytes)
371429 Less than 1 min ago 650 d, 21 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04._DEBIT%2037145-3749_SOL%20MEX_MCTX/06.04.04._DEBIT%2037145-3749_SOL%20MEX_MCTX_.xls (188,416 bytes)
371428 Less than 1 min ago 651 d, 4 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04_mo%2062092_DLIE%20HUB%200C0456_MCTX/06.04.04_mo%2062092_DLIE%20HUB%200C0456_MCTX_.doc (140,800 bytes)
371427 Less than 1 min ago 640 d, 23 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.02.04_so%2062013_HP%20SINGAPORE_MCTX/06.02.04_so%2062013_HP%20SINGAPORE_MCTX_.xls (136,704 bytes)
371426 Less than 1 min ago 650 d, 21 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.04.04._DEBIT%2037145-3749_SOL%20MEX_MCTX/06.04.04._DEBIT%2037145-3749_SOL%20MEX_MCTX_.doc (136,704 bytes)

Recent errors
Visited Reason Url
--------------------+--------------------+-------------------------------------------------------
Less than 1 min ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/05.21.04_so%2061019_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/05.21.04_so%2061019_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.doc
Less than 1 min ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/05.20.04._DEBIT%2037135%2037136%20RMA%204960%204961_SOLECTRON%20MEX_MCTX/05.20.04._DEBIT%2037135%2037136%20RMA%204960%204961_SOLECTRON%20MEX_MCTX_.xls
Less than 1 min ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/05.14.04_so%2061019_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/05.14.04_so%2061019_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.xls
Less than 1 min ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/05.14.04_so%2061018_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/05.14.04_so%2061018_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.xls
1 mins ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/04.21.04_so%2059004%20NO%20CHARGE_HI-TRON-SEOUL%20KOREA%20_MCTX/04.21.04_so%2059004%20NO%20CHARGE_HI-TRON-SEOUL%20KOREA%20_MCTX_.doc
1 mins ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/04.12.04_so%2060028_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/04.12.04_so%2060028_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.xls
1 mins ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/04.01.04_so%2059290_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/04.01.04_so%2059290_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.doc
1 mins ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/03.30.04_so%2059559_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/03.30.04_so%2059559_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.doc
1 mins ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/03.30.04_so%2059510_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/03.30.04_so%2059510_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.doc
2 mins ago Cannot read file:// file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/03.16.04_so%2058579_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX/03.16.04_so%2058579_HP%20SINGAPORE%20CO%20BAX%20RECEIVING_MCTX_.doc

Next Pages to be walked
Next Check Modified Url
--------------------+------------------+-------------------------------------------------------
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2061993_DELL%20ASIA%20MALAYSIA_MCTX/06.07.04_so%2061993_DELL%20ASIA%20MALAYSIA_MCTX_.doc (137,728 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2061993_DELL%20ASIA%20MALAYSIA_MCTX/06.07.04_so%2061993_DELL%20ASIA%20MALAYSIA_MCTX_.xls (188,416 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062195_DELL%20ASIA%20MALAYSIA_MCTX/06.07.04_so%2062195_DELL%20ASIA%20MALAYSIA_MCTX_.doc (137,216 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062195_DELL%20ASIA%20MALAYSIA_MCTX/06.07.04_so%2062195_DELL%20ASIA%20MALAYSIA_MCTX_.xls (187,904 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062196_DELL%20ASIA%20MALAYSIA_MCTX/06.07.04_so%2062196_DELL%20ASIA%20MALAYSIA_MCTX_.doc (139,776 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062275_DELL%20COMPUTER%20CHINA_MCTX/06.07.04_so%2062275_DELL%20COMPUTER%20CHINA_MCTX_.doc (139,776 bytes)
ASAP 648 d, 1 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062275_DELL%20COMPUTER%20CHINA_MCTX/06.07.04_so%2062275_DELL%20COMPUTER%20CHINA_MCTX_ChineseNonWood_.doc (240,128 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062402_FRONTLINE%20TECH%20SINGAPORE_MCTX/06.07.04_so%2062402_FRONTLINE%20TECH%20SINGAPORE_MCTX_.doc (138,752 bytes)
ASAP 648 d, 2 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.07.04_so%2062402_FRONTLINE%20TECH%20SINGAPORE_MCTX/06.07.04_so%2062402_FRONTLINE%20TECH%20SINGAPORE_MCTX_.xls (134,656 bytes)
ASAP 647 d, 3 hr+ ago file://evergreen/corp/MFG/public/Shipping%20Documents%20%28DO%20NOT%20DELETE%29/2005%20Files/PRE%202005-03-24%20DOCUMENTS/COMERCIALS%20MADE%20FOR%20OTHER%20COMPANIES/MCTX%20Laredo/2004/06.08.04_mo%2062351_DLIE%20HUB%2009Y556_MCTX/06.08.04_mo%2062351_DLIE%20HUB%2009Y556_MCTX_.doc (138,752 bytes)

Webinator Walk Report for evergreen_corp

Creating database s:\texisindexes\evergreen_corp/db2...Done.
Walk started at 2006-03-15 13:05:38 (by user)
JavaScript walking enabled
HTTPS walking disabled
Start fetching at file://evergreen/corp/
Ignore urls containing any of the following:
/cgi-bin/
~
?
/private
started 1 new (2264) on file://evergreen/corp/
Process memory limit exceeded (current: 84,094,976, limit: 50,000,000)
3995 pages fetched (578,953,484 bytes) from file://evergreen/corp/
started 1 (2180) Resume 44187392f2
Process memory limit exceeded (current: 51,773,440, limit: 50,000,000)
5225 pages fetched (535,622,485 bytes) from file://evergreen/corp/
started 1 (2952) Resume 44187392f2
Process memory limit exceeded (current: 52,822,016, limit: 50,000,000)
20740 pages fetched (-945,248,622 bytes) from file://evergreen/corp/
started 1 (2772) Resume 44187392f2
Process memory limit exceeded (current: 54,984,704, limit: 50,000,000)
15009 pages fetched (-187,039,406 bytes) from file://evergreen/corp/
started 1 (1924) Resume 44187392f2
Process memory limit exceeded (current: 58,585,088, limit: 50,000,000)
19281 pages fetched (-1,275,984,710 bytes) from file://evergreen/corp/
started 1 (476) Resume 44187392f2
Process memory limit exceeded (current: 51,273,728, limit: 50,000,000)
21741 pages fetched (1,111,021,255 bytes) from file://evergreen/corp/
started 1 (2820) Resume 44187392f2
Process memory limit exceeded (current: 52,187,136, limit: 50,000,000)
9640 pages fetched (555,773,044 bytes) from file://evergreen/corp/
started 1 (708) Resume 44187392f2
Process memory limit exceeded (current: 50,102,272, limit: 50,000,000)
27608 pages fetched (622,905,824 bytes) from file://evergreen/corp/
started 1 (3028) Resume 44187392f2
Process memory limit exceeded (current: 55,324,672, limit: 50,000,000)
35564 pages fetched (-1,720,309,067 bytes) from file://evergreen/corp/
started 1 (2912) Resume 44187392f2
Process memory limit exceeded (current: 62,169,088, limit: 50,000,000)
22893 pages fetched (-757,045,212 bytes) from file://evergreen/corp/
started 1 (3492) Resume 44187392f2
Process memory limit exceeded (current: 50,122,752, limit: 50,000,000)
4604 pages fetched (647,137,726 bytes) from file://evergreen/corp/
started 1 (4000) Resume 44187392f2
Process memory limit exceeded (current: 51,826,688, limit: 50,000,000)
9843 pages fetched (56,974,596 bytes) from file://evergreen/corp/
started 1 (2376) Resume 44187392f2
Process memory limit exceeded (current: 52,756,480, limit: 50,000,000)
7679 pages fetched (-1,615,129,460 bytes) from file://evergreen/corp/
started 1 (3868) Resume 44187392f2
Process memory limit exceeded (current: 63,827,968, limit: 50,000,000)
20879 pages fetched (-187,381,038 bytes) from file://evergreen/corp/
started 1 (1900) Resume 44187392f2
Process memory limit exceeded (current: 63,909,888, limit: 50,000,000)
126 pages fetched (884,882 bytes) from file://evergreen/corp/
started 1 (2920) Resume 44187392f2
Process memory limit exceeded (current: 50,274,304, limit: 50,000,000)
10018 pages fetched (13,573,080 bytes) from file://evergreen/corp/
started 1 (3176) Resume 44187392f2
Process memory limit exceeded (current: 53,993,472, limit: 50,000,000)
5338 pages fetched (11,595,184 bytes) from file://evergreen/corp/
started 1 (2752) Resume 44187392f2
Process memory limit exceeded (current: 50,814,976, limit: 50,000,000)
6897 pages fetched (358,164,749 bytes) from file://evergreen/corp/
started 1 (476) Resume 44187392f2
Process memory limit exceeded (current: 52,973,568, limit: 50,000,000)
10167 pages fetched (1,185,917,395 bytes) from file://evergreen/corp/
started 1 (3804) Resume 44187392f2
Process memory limit exceeded (current: 51,675,136, limit: 50,000,000)
3042 pages fetched (240,837,781 bytes) from file://evergreen/corp/
started 1 (656) Resume 44187392f2
Process memory limit exceeded (current: 55,050,240, limit: 50,000,000)
20026 pages fetched (-1,532,703,643 bytes) from file://evergreen/corp/
started 1 (1212) Resume 44187392f2
Process memory limit exceeded (current: 50,630,656, limit: 50,000,000)
8941 pages fetched (43,981,677 bytes) from file://evergreen/corp/
started 1 (404) Resume 44187392f2
Process memory limit exceeded (current: 50,319,360, limit: 50,000,000)
6722 pages fetched (1,736,291,705 bytes) from file://evergreen/corp/
started 1 (2588) Resume 44187392f2
Process memory limit exceeded (current: 51,650,560, limit: 50,000,000)
25851 pages fetched (-2,005,530,236 bytes) from file://evergreen/corp/
started 1 (4068) Resume 44187392f2
Process memory limit exceeded (current: 52,903,936, limit: 50,000,000)
7251 pages fetched (809,544,294 bytes) from file://evergreen/corp/
started 1 (3856) Resume 44187392f2
Process memory limit exceeded (current: 53,993,472, limit: 50,000,000)
34650 pages fetched (6,318,247 bytes) from file://evergreen/corp/
started 1 (432) Resume 44187392f2
Process memory limit exceeded (current: 56,270,848, limit: 50,000,000)
5151 pages fetched (717,752,690 bytes) from file://evergreen/corp/
started 1 (2772) Resume 44187392f2
Process memory limit exceeded (current: 56,635,392, limit: 50,000,000)
6680 pages fetched (1,575,819,175 bytes) from file://evergreen/corp/
started 1 (3936) Resume 44187392f2
Process memory limit exceeded (current: 50,225,152, limit: 50,000,000)
10388 pages fetched (-2,128,982,848 bytes) from file://evergreen/corp/
started 1 (3908) Resume 44187392f2
Process memory limit exceeded (current: 59,138,048, limit: 50,000,000)
19290 pages fetched (433,933,734 bytes) from file://evergreen/corp/
started 1 (3540) Resume 44187392f2
Process memory limit exceeded (current: 52,297,728, limit: 50,000,000)
14912 pages fetched (-602,226,871 bytes) from file://evergreen/corp/
started 1 (2788) Resume 44187392f2

Post by **John** » Fri Mar 17, 2006 12:50 pm

Which version of the scripts do you have installed?

It looks as if it keeps running out of memory, and
restarting. It is possible that if you have some older scripts it has in effect started refreshing, so you could do a Pause and Live to make the current set live.

scott.shaver · Post by **scott.shaver** » Fri Mar 17, 2006 12:54 pm

Webinator 5.1.29-Windows-w/plugin

Webinator 5.1.29
$Id: search4.src,v 2.230 2006/01/11 17:50:33 kai Exp $
thunderstone_file_sha1: 91ca173093691fd63c360e717d882c9b9926289f

Webinator 5.1.29
$Id: dowalk.src,v 2.408 2006/01/11 22:28:33 mark Exp $
thunderstone_file_sha1: e4ce19a8e5792eb03c9bff20393b337e9f3b8024

I've got the Maximum Process Size set to Medium, should I up it? The machine has 2 gigs of ram and 2 processors that are duplexed to look like 4.

scott.shaver · Post by **scott.shaver** » Fri Mar 17, 2006 1:04 pm

I upped the size to Large. I click pause walk and live.

it says it is running:

E:\MORPH3\texis.exe -r profile="evergreen_corp" "E:\MORPH3\texis\scripts/Webinator/dowalk\hold.txt"

When I then go to the walk status page it still seems to be running the walk. Do the monitor processes have to quit before the walk stops? There are four of them in the task manager.

scott.shaver · Post by **scott.shaver** » Fri Mar 17, 2006 1:20 pm

LOL, now I have 5 monitor processes each taking no CPU time. A texis.exe process that is chewing up memory like crazy, taking 25% CPU. Another Texis process that starts about every 5 seconds and immediately ends. And the wlak still seems to be running.

Post by **John** » Fri Mar 17, 2006 1:46 pm

What does the bottom of the walk status show? The one using 25% (1 of the 4 CPUs) is probably actually building the index on the crawled data.

scott.shaver · Post by **scott.shaver** » Fri Mar 17, 2006 1:52 pm

It finally stopped.

Creating search index on fetched pages...Done.
Creating spell-checker dictionaries...Done.
Done.
Verifying usability of new walk.

Walk finished at 2006-03-17 11:41:12 (took 45 hours 51 minutes 41 seconds)
Making new database live: s:\texisindexes\evergreen_corp/db2

--------------------------------------------------------------------------------
Checking for broken hyperlinks...

Now I have 3 monitor.exe tasks running and no texis.exe. Should I kill them?

Alos do you think I should leave the process size set to Large and start the walk again?

The live search is now showing tons of results, execellent. Must have been the index not created yet.

Post by **mark** » Fri Mar 17, 2006 2:28 pm

Don't kill monitor.
Leave the process size at large.
Make sure your rewalk type is set to refresh before starting it again so you don't have to start all over. It will pickup where it left off when in refresh mode.