Delete URLs from database takes hours

Post Reply
nroot
Posts: 8
Joined: Tue Oct 24, 2006 3:16 pm

Delete URLs from database takes hours

Post by nroot »

Hi- I've got a profile with ~200,000 URLs in it and I want to remove ~15,000 of them (I've create a separate profile to manage them). So I went to the list/edit URLs page and specified a "delete" of a pattern like "http://server.com/directory/*". On a 4 processor server with 4GB of RAM, this action takes more than 2 hours and slows everything down to the point where most live searches timeout. The action *does* finish correctly eventually. It doesn't always take this long for me to delete URLs -- seems to just be associated with this particular profile.

First question: is there any way to check progress and/or stop one of these delete commands once it's going (if it looks like it's crushing server performance).

Second: Am I doing something wrong here? Is this normal performance?

Thanks in advance for any help! Best - N
User avatar
mark
Site Admin
Posts: 5519
Joined: Tue Apr 25, 2000 6:56 pm

Delete URLs from database takes hours

Post by mark »

Yes. I've see it where deleting can take substantially longer than listing the same pattern. Deleting does more work keeping indices up to date and deleting associated information like links to other pages (but not the linked pages). If the pages have a lot of links deleting the refs can take longer than deleting the page itself.

There's not really any way to check progress or reduce the load. Though the delete process would only use one processor and not a lot of memory. It will be pretty heavy in disk I/O though.
Post Reply