Ghostfiles Service http://www.lowrieweb.com/downloads/ghostfiles.exe 3.1.4.3 Enhancement Improved look ahead Lookup mechanism can cause a cumulative effect under extreme conditions. A new mechanism to look ahead in the event queue and remove redundant events has been implemented. Changes to DropEvent and implementation of DropAllModified Bug Fix Cache mechanism can cause files in source to be deleted Under rare conditions, the cache file can cause files in the source to be erroneously deleted. This happens when the cache is strongly out of line with the directory. Fix applied to cache mechanism to recognize these conditions. Bug Fix Error updating a read only file When updating a read only file, the copy process failes with Access Denied when an attempt is made to update the destination file. The problem is the destination is readonly. A check is now being made in the copy for readonly destination. If found, the readonly flag is removed before copy and reinstated after copy. Bug Fix Entry Names with Single Quotes A previous fix allowed names with quotes to be saved. However, a bug in ENTRIES.HTM caused these entries to be unmodifyable - edit and delete. Change to SELECT code now enables quoted names to be selected, edited and deleted. Bug Fix Hidden files and directories Ghostfiles fails to correctly replicate hidden file attributes and ceases synchronization on copying empty files. Fix to Sync Code Bug Fix Auto Update Minor fix for two digit version numbers Fixed - requires new sub minor version. Bug Fix 100% CPU when synchronizing. Bug during synchronization when a file matches the excluded files list. Ghostfiles enters a tight loop, using 100% CPU and never returning. Include and Exclude file checks now work correctly. Bug Fix Files are not correctly truncated during copy when the source is smaller than the destination. This bug was introduced in version 3.1.3.7 as part of the remote source optimization. SetEndOfFile() reintroduced to file copy function. Enhancement Ghostfiles detects if source is on local machine. If not, CRC32 optimization is skipped. CRC32 optimization works well for local source files but is expensive for large remote source files. This enhancement allows Ghostfiles to skip CRC32 optimization for remote sources. Bug Fix Critical Bug Fix - Optimizing compiler problem causing unexpected application failure. This is an initial fix to an unexpected application failure. We suspect the issue is related to optimization settings in the compiler. Remove CString reference parameters to format strings and compiler optimization flags. Enhancement Improved GFDUMP Ghostfiles now contained an enhanced crash dump facility. Bug Fix Fails to remove transient files from file history when synchronizing During synchronisation, if a file in the file history is not present in the source, the file should be removed from the destination and from the file history. Ghostfiles failed to remove the file from the file history. Files are correctly removed from file history during sync process. Bug Fix CRC32 calculation bug A bug in CRC32 calculation resulted in some changes not being detected when CRC32 tested. Fixed CRC calculation in Entry processing. Bug Fix Improvements to file copy - fixes low performance and high memory use when processing large files. The previous file copy scheme uses memory mapped files. These perform well for small files but with large files, CPU and memory usage could exceed system limits. This could lead to crashes. The scheme has been modified to eliminate the use of memory mapped files. Further improvements have been made to the CRC32 calculations to enable small file operations to perform well. Enhancement File List Management Minor enhancement to error handling around file list management. Bug Fix GFNOTIFY Registration Removed extra backslash when registering GFNOTIFY Fixed in Installshield setup. Enhancement Auto Update Feature Ghostfiles now checks for updates once a day. If an internet connection is available, updates are checked and if available, an update tab is shown on the main screen. Enhancement Added ClearSyncFile command line function ClearSyncFile is used to clear the sync files used by entries. This function should only be called if you are experiencing problems with synchronization. Enhancement Added DumpEvents documentation to Help File Added DumpEvents documentation to Help File. Added additional 'code safety' to DUMPEVENTS implementation Enhancement Using High Resolution UTC time Ghostfiles' events have now been converted to use a high resolution universal time to record their event time. Enhancement Events XML Dump Added a command line argument DUMPEVENTS to create an XML file enumerating all cutrent pending events. Bug Fix Failure to replicate Word Documents Ghostfiles frequently failed to replicate changes to Microsoft Word Documents and complained of 'Transient' files. Fix to handling of renamed files when source or destination are missing. Enhancement Disk Quota Management When replicating, Ghostfiles is able to monitor the destination free space. If a set limit of disk is used, Ghostfiles deletes the oldest files from the destination. This function is not available if full mirroring is enabled. Bug Fix Deleting an entry failes to remove Sync File setting from registry When an entry is deleted and Ghostfiles is restarted, the sync file setting is recreated after the registry entry is removed. This results in the entry partially remaining in the registry and on restarting Ghostfiles attempts to load this entry. A warning is then shown as the entry settings are incomplete. Removing an entry now unwatches the entry and deletes the settings in the registry correctly. Enhancement Improvements made to the file list / CRC32 management Improvements to the size of the file through a Least Recently Used mechanism. This is important when replicating large file sets. Bug Fix Entry watching fails to terminate when an error is encountered for a replication between local source and network destination when using full mirroring The consequence of this bug is the system never recovers from the network error. Timeout on monitoring thread now terminate the thread. Bug Fix Modify event fails to trigger network recovery condition When a change to a file (modify) is encountered, if the destination is unreachable, the function fails to interpret this as a recoverable condition. Network error detected and correctly reflected back by this function Bug Fix Missing entries in SITE.CSS caused statistics view to fail to render Erroneously saved SITE.CSS with two entries missing. These entries are used by the VML code in the statistics view. As a result, the stats views no longer rendered Put back original SITE.CSS Bug Fix Further bug in list management Causes a null pointer to be referenced in the main entry processing loop. Fixed. Bug Fix Bug fix similar to previous bug fix in list management. An in memory page fault occurs very occasionally when Ghostfiles encounters a corrupt source or destination. This can cause the service to stop. This fix traps and recovers from this fault. Bug Fix Severe bug in list management Severe bug in list management causes the service to terminate unexpectedly. This happens on rare occasions when files are created transiently. Changed the list management to use a fixed array. Tested for 24 hours continuous use. Enhancement Added crash dump facility In the event of an unexpected crash or termination of the service, Ghostfiles now produces a crash dump file called GFDUMP.LOG. If this file is created, please contact our support team at support@lowrieweb.com and attach the log file together with any other information you have. Bug Fix Locked file in source fails to copy if destination doesn't exist This happened through using CopyFile() for missing destination file. However, this function requires unlocked access to source. Using Ghostfiles' copy function Bug Fix File masks not initialized when synchronizing File mask initialization is called when watching directory - this is after the Synch Process - hence Sync does not take not of file masks. Moved file mask code to Entry initialize. Enhancement Launch Help and Lowrieweb in new window Selecting 'Help' or 'Lowrieweb' from the menu launches these pages in a new browser window. This allows users to refer to the help without loosing current changes. Bug Fix Creating multiple sync temp files Latest version used temporary files in the application directory to record the state of the entries source directories. A bug in this process resulted in temporary files being recreates on each run of Ghostfiles. Error when writing to registry, now fixed. Enhancement Support for Local User Settings By default, Ghostifles runs as a service and uses HKEY_LOCAL_MACHINE. When run from the command line using the -debug option, Ghostfiles can be passed a -currentuser argument. This argument forces Ghosfiles to use the local registry (HKEY_CURRENT_USER) and to use named entries in the registry. See help file for more details (Ghostfiles Command Line Settings). Enhancement Email now has a FROM address and FRIENDLY name Email notifications now require a from address. This allows larger users to identify the sender of the alert. Enhancement 30 day unregistered Ghostfiles now operates for 30 days (not 4 hours) when unregistered. Once timed out, the user is automatically directed to the timed out page. Enhancement Improved Synchronization Ghostfiles now keeps a 'sync log' for each entry. This file holds details of the files on the source (CRC32) and is used to reduce the work over the network required when synchronizing. This provides a significant improvement to the performance during replication and synchronization and is especially valuable when working across slow network links (such as VPN). Enhancement Special handling for files over 4Gb in size All file handling in Ghostfiles is restricted to 32bit and hence files over 4Gb are not supported. These files are now trapped during the file notify and synchronizatio process and a warning is generated. The help file has been updated. Enhancement Network error when starting replication is not currently treated as a recoverable error. Handled in this version Added SetInError to WatchDirectory() function. Bug Break in a VPN network connection not detected. Failed to trap BAD NETWORK PATH. This has been added and VPN connection breaks are now treated as a recoverable network error. Modified in InNetworkError() function. Bug Exclude file spec does not work for renamed files This was investigated and although working as specified, the algorithm was changed to create the file if renaming from exluded to included file spec and delete the file if renaming from included to excluded file spec. Rename handling changed for this fix. Bug Failure to replicate to destination Problem caused when forming the destination file spec from the source file spec. This occurs due to a 'full mirror' test that was introduced. The function responsible for building the destination file spec has been reviewed and fixed. Bug Error recognizing include file spec * and *.* Include file spec function fails to recognize certain file (file names with spaces) with wildcards * and *.* Added check for * and *.* Bug Large file copy causes bogus error in destination. Although the copy takes place, there is a Failure error produced. This error has a zero GetLastError() code Error handling locked files (happens when large files are copied). This error is now recognized and handled through rescheduling the event with a 10 second delay. Bug Fails to launch browser from task bar when run as a network account service. When set to log in as a service to a network account, the task bar icon fails to launch the browser. The launch hangs. Problem due to window station associated with network account. Solution has been to re-work taskbar notification. This is now is a separate app called GFNOTIFY.EXE - run at startup. This works well in all conditions. Bug About box (gfservice -help) fails to display any text. Changes to LADlg base class now requires a call to SetColor(). This was missing, hence, no colour for text and background. Added call to SetColor() Bug Bug saving new entries Introduced a bug when saving a new entry. Often used the entry name of a previously selected entry. Fix to ModifyEntry() function - when saving in registry, use name rather than session name for registry key. Bug Sleep in file copy causes a performance problem Sleeps(0) were introduced to ensure co-operation during large file copy. However, this caused a major less of performance. Sleeps have been removed and we are trialling new copy/move functions. Bug Handle Leak after restart During the process of restarting the service, two thread handles were leaked. This causes a degradation in performace. Using AfxBeginThread rather than CreateThread Bug Browser Page Refresh causes whole page to refresh. This causes the page to re-load and scroll is returned to top of document. This feature makes viewing graphs and progress difficult. Fix to Javascript to reload using history(0). Bug XML Log can fail to write XML header to log file. The result is the XML log file is malformed and not renderable by the XSL. Occurs when file is renamed to old - header is not prepended. Also set the max file size to 1Mb Bug Ghostfiles limited to 15 Entries Ghostfiles can only support 15 running entries. NOT REPRODUCABLE - Tested with 25 live/running entries. Enhancement Full Mirroring Full mirroring means the source and destination are mirrored in both directions. This should be an optional setting on the Entry. Bug Failure to shutdown during synchronization While synchronizing, although the service responds to a shutdown, the shutdown only takes place once synchronization has completed. This is a problem when synchronizing a large data set. Entries Shutdown was waiting for sync to finish rather than signalling sync to finish. Enhancement Statistic View Additional menu option showing bar graph statistics on entry events, errors and warnings. Enhancement Sync on recovery from Error Entry option. If selected then on recovering from a network/source/destination error, the destination is automatically synchronized with the source. Enhancement Enhanced Online Help Using a table of contents view for easier navigation Enhancement Display during Synchronisation Enhanced to show more progress during synchronisation. Very important when synchronising a large data source. Also, synchronisation copy process has been made more co-operative. FAQ updated to describe synchronisation verses replication. Enhancement Error notification Email notification when Ghostfiles errors occur. Notification would be set at the 'Settings' level. All error level events would then be sent to an email recipient. Enhancement Scheduled Re-Sync Schedule to re-sync files between source and destination. The schedule allows an entry to perform a re-sync of all source files to the destination. The schedule is based around 'every day at' and 'every period' Implemented through command line arguments. Once running, a second instance of gfservice is used to pass an command message to the first running instance. Currently five commands are supported, including 'sync'. This can be called from a batch file or from the Windows Task Scheduler. Fix Source files become locked by Ghostfiles In rare events when the destination is inaccessible during a file replication process, the handle on the source file may be left unclosed. The result is that the source file becomes locked until Ghostfiles service is shut down. Fix to copy routine to release source file handle if destination file access fails. Enhancement XML Log file Log file now created in XML log format. Product ships with XLS file to view the log. The log file size is also managed. When logging, any log files greater than 100Kb are archived before starting a new log file. Enhancement Add 'Exclude Files' feature Wildcard specification that allows files of selected file format to be excluded from replication. Bug Fix Failure to replicate directory operations When creating a directory, the destination is only created when files are created on the source. When deleting a source directory, the destination files are deleted but the directory remains. The work around is to use the wild card '*' as part of the entries wildcards. Added test for directory when testing wildcards against replication event Enhancement Survive Network Errors Feature to poll for network connection after a failure. If network is re-established, the replication should be restarted for that entry. Enhancement Replicate to an FTP destination Entry setting to replicate files to an FTP site destination. Enhancement Alternate security settings against entries Entry option providing an alternative (to service settings) user name and password for accessing the destination resource. Enhancement Replicate timestamps and permissions Source directory permissions and timestamp should be replicated to the destination. This should be done under flag control. Enhancement Move files Under flag control, files for an entry are moved rather than copied. Enhancement Archive Feature Files are replicated to date/time stamped directories. Enhancement Don't replicate when files are being accessed by Anti-Virus software. Certain anti-virus software marks files as modified after the files have been scanned. This causes Ghostfiles to replicate the file. This enhancement ensures Ghostfiles does not replicated unchanged files scanned by anti-virus software Enhancement Control over page refresh rate. Add maintenance and control over page refresh rate. Currently fixed to 20 seconds. Enhancement Log to file New setting and function to log main screen progress to file. Bug Fix Failed to find path error Error message shown when synchronising on start-up. Bug in synchronisation thread fixed. Bug Fix Wildcards not working Error caused by changes to delayed directory replication. Added wildcards to include/exclude filter. Bug Fix Moving Directories Fails When a source directory is moved to or from a sub-directory, this operation fails replication. Added directory watching to replication logic. Enhancement Synchronization in sub-thread To improve performance, synchronisation should be threaded Bug Fix Network error causes replication to terminate When replicating over a network, any errors during replication associated with network faults causes replication to terminate silently. This condition is now trapped and reported. Replication requires a restart after fixing the network error. Ghostfiles requires a future enhancement to handle network errors and restart replication on network recovery. Also requires network monitoring. Enhancement Delete before synchronisation Added function to perform delete on destination before synchronisation. This provides a 'true' synchronisation of the source to the destination. Enhancement Service Stop and Restart from Taskbar Add control over the service – service stop and service restart . Bug Fix Taskbar icon available when running as non-interactive service When the service is logged into a network account, the service is not able to interact with the desktop. Using workstation context to drive the taskbar user interface thread. This shows the icon regardless of service account status. Bug Fix Synchronisation fails to account for sub-dir flag Synchronisation does not take account of the sub-directory flag and always synchronises the entire source. Modification to synchronisation thread to take account of sub-directory flag Enhancement Major improvements to performance Many changes to improve performance. There is now no delayed handling when replicating. All handling is now through managing an event list with time stamped elements. Bug Fix Synchronisation does not handle multiple wildcards When using multiple wildcards in the form '*.EXT1;*.EXT2', this is not recognised during synchronization. Modification to synchronisation thread to test wildcards rather than using wildcards in FindFile logic. Bug Fix Won't shutdown while busy replicating. When busy processing replication events, the system is unresponsive to service shutdowns Service running status is monitored at frequent points within the file copy functions Enhancement Added queue size status Added an indicator on the main screen showing the number of pending replication events in the replication event queue. Bug Fix Memory leak Memory leak detected when replicating. Minor leak detected, related to task bar management. Memory leak located and fixed.