A new set of appliance updates are available: texisScripts-25.0.2, applianceman-25.0.2, texis-8.01.1667348496, thunderstonePatch-2.20.
texisScripts-25.02 provides these features and fixes:
- Admin accounts are now always listed in alphabetical order
- Creating a profile can give getDataspaceDir ... FOP_EDOMAIN error if a /data* mounted filesystem is smaller than 1GB total, and profile creation may prefer smaller such disks for dataspace too much; fixed
- Exponential notation could appear for some floating-point numbers greater than 10^6 (e.g. populating a Parametric Number field, Maximum Refresh Time); now only used if value is greater than 10^12, and 12 digits of precision instead of 6 are used
- Fixed DFS file server walks failing to start up in primer system
- Your IP: in System Information's Client section now properly take X-Forwarded-For into account
- <meta property> and <meta itemprop> tags now available in walks (e.g. for Data from Field)
- Last-Modified and Expires now obtained from <meta property> and <meta itemprop> if header, <meta name>, nor <meta http-equiv> available. Same for Keywords and Description, which additionally come from <meta property="og:..."> if not found elsewhere. Test Fetch now shows <meta property> and <meta itemprop> values.
- Fixed Search Timeout setting not affecting metasearch frontend's fetch timeout to backend targets
- Support Commands now receive additional context from the running system
- Modifying best bets on List/Edit URLs Details pages now uses the modern, undoable interface
- Fixed Test Fetch of http-post:// URLs not performing the actual POST http transfer
- Fixed replication when restoring a backup of profiles that already exist
- Best Bets are now included in Settings replication
- Removed ability to rename bestbet groups, necessary for best bet replication
- Added ability to create ACLs for Static Content and Client Certificates
- Added ability to create ACLs on admin-created directories inside Static Content
- Added highlighting to every other row in Profiles list
- Diskspace Viewer and Cleanup now properly support appliances with multiple dataspaces
- Fixed Test Fetch not applying the same link sanitization that walks would apply
- Test Fetch option Fixed font added, to print all values in fixed-font and preserving spaces. Wrap long lines now also does so in large values like Text
- Added Keep Selectors, Ignore Selectors to keep or ignore HTML elements' text via CSS selectors; defaults to ignoring <nav>, <header>, <footer> elements. Keep Tags renamed to Keep HTML Strings to clarify what it searches for (simple text strings).
- Keep HTML (Alt Text, <STRIKE>, <DEL>, <FORM>) settings removed; can now be accomplished with Ignore Selectors. Existing profiles with non-default values for these settings will have equivalent Ignore Selectors values added.
- File crawls now get File-Attributes header with Windows file attribute tokens
- Added best bets support to dataload API
- Send Profile Settings in Replication Tools now sends Best Bets along with profile settings
applianceman-25.0.2 updates the documentation.
texis-8.01.1667348496 provides these features:
- Fetches/crawls now use OpenSSL 3.0.7; fixes CVE-2022-1292, CVE-2022-1343, CVE-2022-1434, CVE-2022-1473 (these do not affect appliances' use of OpenSSL 3.x). Also fixes CVE-2022-3602 and CVE-2022-3786: neither affect appliances: while texis RPM packages from 2021-10-11 until this update use OpenSSL 3.x < 3.0.7 (affected by the CVEs), it is only used for crawls, and no remote certificates are validated during crawls.
- File types plugin (anytotx) now shows <meta http-equiv,property,itemprop> values (in addition to <meta name>) from HTML documents
- Formatted text from HTML containing <figure>, <svg>, <canvas>, <article>, <aside>, <figcaption>, <footer>, <header>, <main>, <marquee>, <nav>, <section>, <details>, <summary>, <command>, <ruby>, <audio>, <video>, <template>, <slot>, <spacer> tags may change whitespace, e.g. space to newline(s); re-walked unchanged pages may thus change hash and appear to be changed
- Some HTML formatter changes involving priority of <div>/<span>/<table>/<tr>/<td>/<th>/<tfoot>/<thead> (and some others like <optgroup>); could possibly affect whitespace in formatted text and thus change crawled pages' hashes
- <img alt="..."> text, and widget whitespace (e.g. double newline for <p>), were still being printed when they should be suppressed (e.g. inside <style>...</style>, ignoretextselectors/Ignore Elements matches, etc.); fixed. Single space is still printed, to maintain separation of potential bracketing non-suppressed words. Caveat: fetched pages' content (including whitespace) may change, and thus hashes when walking.
- Upgraded to exiftool version 12.47: gets JPEG comments for anytotx plugin
thunderstonePatch-2.20 provides some system updates, and is needed by
texis-8.... RPMs.
Note: Some older updates (listed below) were released late; their functionality is also part of (or superceded by) the above updates. These include:
texis-8.01.1651095857, which provides the following (part of or superceded by the above
texis update):
- Updated fetch lib to OpenSSL 1.1.1i to fix CVE-2020-1971
- Updated unrar to 6.0.3 for fixes
- ExifTool upgraded to 12.30
- Updated fetch lib (for crawls) to OpenSSL 1.1.1l to fix CVE-2021-3711 and CVE-2021-3712, neither of which affect Texis nor crawls
- Some robustness improvements to DNS lookups, especially under Windows
- With Best Bets enabled, a user query containing one or more consecutive dashes or plus signs with space or begin/end of query to both sides, might cause an ABEND; fixed. Workaround is to disable Best Bets, or pre-process the query to avoid such queries.
- Crawls/fetches now use OpenSSL 3.0.2; fixes CVE-2021-4044 (might cause SSL connects to fail; affects Texis versions from 2021-10-11 through this fix) and CVE-2022-0778 (possible infinite loop during cert checks; should not affect Webinator/Appliance generally as client certs not checked during crawls)
- Texis version 8.01 released:
- IPv6 crawling support added (with texisScripts 24.0.1 or later)
- Longer admin passwords supported, more secure hashing (on password create/change after this update). Reboot recommended (after stopping all walks) to ensure password table is updated to enable this feature.
- Under Linux, taskmonitor process can get stuck waiting for a child process that appears to be (from ps cmdline args) a copy of itself; fixed
- Fetching a page that redirects may cause an ABEND; fixed. Affects versions from 2020-09-28 and later, prior to this fix.
Note: During application of this update, the GUI might refresh to a Script not found or Vortex error page. If this happens, wait 30 seconds and re-load the page. The error should be transient and benign.
texisScripts-24.1.0, which provides the following (part of or superceded by the above
texisScripts update):
- Fixed viewing performance log showing display name for token [dataFromField] not found
- Fixed URL deletions with Ignore Case=Y not matching in some scenarios
- Improved walk efficiency when indexing multiple servers in the same profile
- Fixed restoring backups of existing profiles with per-profile replication settings sometimes using wrong replication settings
- Fixed race condition where a replicated Finalize Walk that fails during initialization could have no pid set in its task list entry
- Fixed rare race condition on Walk Status when the status page starts rendering before a walk starts up, and the walk finishes before Walk Status finishes rendering
- Fixed replication of refreshed URLs that weren't modified not replicating their meta data correctly
- Clicking on a URL in List/Edit URLs results might have brought up an empty-URL item instead of the desired-URL item, if an empty-URL item existed (e.g. due to Dataload with empty URL); fixed
- Some long-running daemons (task monitor, Results Cache manager) now exit sooner when commanded to or change detected
- Now defaults to Off-Site Components=N in newly created profiles
- Fixed Show Authorization Info not clearly showing cached Authorization URL checks
- Fixed search XML output displaying an erroneous through of X summary box when displaying past the end of results
- Fixed restoring backups with profiles and thesauruses not getting assigned to the profiles properly
- Fixed replication sender possibly exiting just as items are added to the replication queue
- Improved behavior when search user is provided malform redirect URL that attempts to execute JavaScript
- No longer allows admins to turn off HTTPS server while using HTTPS to access the admin interface
- Login Verification URL can now be used with Forward Login Cookies Results Authorization type
- Deleting a profile now properly stops all tasks running in the profile first
- Admin interface will now check if custom search XSL Stylesheets are well formed, and will not save them if there are errors
- No longer misleadingly allows modifying Parametric Fields on metasearch profiles
- Requesting an admin page when not logged in will now redirect to that page after logging in when using HTTPS
- Detection of missing internal data version is now properly fixed
- Fixed incorrect merging of spelling suggestions in metasearch
- Improve reliability of automatic software updates
- Enhanced refresh algorithm to allow attempting duplicate URLs less often
- Admin Banner System-Wide Setting was not being shown on some admin pages
- Enhanced metasearch merging of spelling suggestions to provide more accurate suggestions
- Fixed Keep/Ignore Tags causing some unusual pages to leave html tags in their content
- Fixed archive log cleanup somtimes removing newer log files before older log files
- Updated JQueryUI to 1.13.1
- Fixed javascript dropin using customer site's jQuery with jQuery.noConflict() logging the error '$ is not a function'
- Version 24.0.1: Support for IPv6 crawling added (in conjunction with texis 8.01 or later RPM)
- Changing admin password was failing to change Webmin password and was giving exec command sudo returned exit code 1 error; fixed. Only affects scripts version 24.0.1 and earlier in combination with Texis version 8, so no appliances should be affected.
Note: All of these updates are currently only available for Gen3 (2013) and later appliances, pending a thunderstonePatch update for older appliances.
Updates can be found via the
System -> System Setup -> Update Software menu item.