We did it that way originally because Webinator 1.0 was written in 'C'
and not Texis Web Script and thus was not as easy to modify. There are
some performance considerations as well.
Each additional SQL clause you add does have an impact on search speed.
If you had a database with 4-5 million pages in it you would probably
not want to add the time cost of the additional LIKE clause.
BTW: The reason you had an issue with the (/dir1,/dir2,/dir3) LIKE clause
was probably due to the index expression(s) used when you created
the Metamorph index on the Url field. See the docs / tech-support db
about "indexing properties" for info on changing the expression.
You could do that, but it would be really expensive computationally.
Perhaps a better solution to this entire problem is to add a Category column
to the the HTML table, and then assign one or more categories to each
of the Urls where appropriate. Then the usage of the Metamorph index on
that field would be more direct. It would also allow you to have a page
reside in multiple categories without issue.
To experiment with this you could make alternate use of the html.Meta field
with a series of SQL commands to set the categories.
eg:
"update html set Meta=Meta+' someword' where Url matches 'www.mysite.com/dir%'
Then add an "and Meta like 'someword'" clause to the search query.