gedcom export error

[wingfield 0]

folks,

I'm getting an error on gedcom export.  I've never seen this before.  I'm running the latest version of TMG 9.05 and have windows 10.  I've attached the error message.  I'd appreciate any help on this.

david

[Max 0]

I could solve this problem by deactivating the antivirus programs before starting the export.
Max

[John Cordes 0]

 Reporting what is basically the same problem, but the standard solutions don't appear to be working. I made a similar post yesterday on the TMG mailing list, and I have been encouraged to post here also.

TMG 9.05 Gold; Windows 10 Pro 64-bit

(I regularly run Optimize-VFI-Optimize-Backup; VFI has not reported any errors for a long time.)

 I often want to export a gedcom file from my database of close to 26,000 individuals (no internal exhibits); at least one reason is to use the gedcom file with GENViewer v2 which, unlike v1, doesn't read the  TMG database directly.

 I have had no problem with this for several years, but following the last major Windows 10 update to v1809, which happened for me about four weeks ago, problems began.

 The issue is that some sort of file locking conflicts apparently take place in the temp directory used by TMG during the generation of the gedcom file. Here's an example of that directory location on my system: C:\Users\Owner\AppData\Local\Temp\TMG71892382

 The Temp directory is deleted if TMG exits normally, so is usually not seen.

  For me, if I click on the Retry button, as shown in the image in the previous post in this thread, (and repeat if the error comes up again) TMG will eventually finish, generating a gedcom file (which is not a valid one -- too small and also not ending with "0 TRLR") and also  deleting the temp directory it used. If either the Abort or Ignore buttons is clicked TMG essentially hangs and requires Task Manager or equivalent to kill the process. I say 'essentially' because the program is not totally locked up. I think the only menu option which isn't greyed out is the 'Center the current window' item. That much does work, but then nothing else can be done.

 I realize that the standard solution here is to say that I have either (a) some antivirus program interfering, and/or (b) a cloud backup program which is monitoring some TMG (or other relevant) directory.

 I have tried hard to make sure that neither of those explanations is at work here. Of course I cannot guarantee that, but I have ensured that BitDefender has exceptions for both the TMG program,  and is also excluding all TMG directories, both the program directory and also the directories where my TMG data resides. I do have cloud backup programs (named below) but they are also marked to not be looking at any of the TMG  directories. I have also tried an export with BitDefender totally disabled and the same error crops up.

A few other observations:

There have been a very small number of occasions where the export *did* work. It's just enough to make me keep trying occasionally. 

Sometimes the Export job can get tantalizingly close to finishing before the error crops up. I think on one occasion it completed all the Individuals and was into Families before it stopped. Possibly a database which could finish in, say,  8 to10 minutes or less would have a larger fraction of successful runs than I am seeing.

 I am not using OneDrive.
 

My TMG data (and the location of the exported gedcom file) is in C:\TMG and that entire directory is excluded from BitDefender and from both my cloud backup programs, Backblaze and IDrive. 

I do run manually some other backup programs, on specifically chosen directories. To the best of my knowledge nothing else of any significance has changed around the time of that Windows update.

I have found only three solutions, or work-arounds, for this issue:                                                                                                                                                                                                                  

1. transfer the TMG backup to an ancient (2005!) XP laptop. The export takes 2+ hours -- I would have to be very desperate to use this.                                                                                     

2. booting in Safe Mode allows TMG to complete the export. Unpleasant (to have to reboot the machine at least twice), but it works.

3. run TMG under Wine on a Linux box. I can copy my TMG files from the Windows machine using rsync and ssh (under cygwin) to the Linux machine. Wine runs TMG fine, provided the version of Wine is recent  enough, and the gedcom export is actually a bit faster than on my Windows computer (about 13 minutes in Linux compared to around 15 - 17 minutes on Windows). I then just have to copy the exported gedcom back to Windows  and it's done.

 Solution 3 is quite workable for me (better than #2), though of course annoying to have to do it.

 My son Peter and I did some monitoring of what was going on in that Windows Temp directory during the gedcom export. He is more up on that sort of thing than I am, but it seems that there was an enormous amount of file opening and closing going on, repeatedly, with a small number of files. It's no wonder that TMG is so slow to do a gedcom export.

 Peter monitored the underlying processes while Wine was doing the gedcom export on the computer running Linux. He noted that under Linux the system calls which were causing all those file open / close operations were  mostly by-passed (at least not repeatedly opening and closing physical disk files).

I am posting this mainly to see if anyone else has encountered an issue like this following that last major Windows update (to v1809) -- or even better, have another suggestion I could try to solve the problem. 

It was strongly suggested on the mailing list that this issue is due to something specific to my machine, and I agree that is very likely the case. It's just that I have not been able to determine *what* that specific something is. Very frustrating!

John Cordes

 

 

 

 

 

 

[Jim Byram 0]

You have tried disabling the BitDefender real-time scanning temporarily and then trying the export?

[John Cordes 0]

Jim - Yes, I disabled BitDefender in its Settings screen and tried exporting a gedcom file. Quickly got the same error appearing.

[Jim Byram 0]

If you have no program running that is affecting files (security, syncing, real-time backup), I have no idea what is blocking the GEDCOM export.

I'm currently running Win10 (17763.379) with Windows Defender and have never had a problem exporting a GEDCOM (in the past or now).

[John Cordes 0]

Thanks for looking into this, Jim. I will continue to try searching for anything which might be monitoring any of the relevant directories in any way, but at the moment I'm out of ideas, unfortunately. 

 

 

[John Cordes 0]

 A follow-up on my gedcom export problem.

  After spending hours with Process Monitor, following the action in the TMG Temp folder where the export process (mostly ?) takes place, setting various filters to make the monitoring manageable, I have found that my IDrive backup program's service (id_service.exe) was tracking all the activity in that folder. 

 A reminder: the Temp folder is located (for me) here:
C:\Users\Owner\AppData\Local\Temp\TMG*****

 I have IDrive set to *exclude* the entire
C:\Users\Owner\AppData\Local\Temp folder.

 Despite this, its backup engine id_service was monitoring activity in the TMG  Temp folder and resulting in Sharing Violations with TMG.

 By stopping the ID service I have now done two successful Gedcom exports in a row. I will try more later today and hope for the best.

 I will contact IDrive about this, but I suspect I will be looking for another program at some point. The workaround of temporarily stopping the service is not a huge issue, but I
shouldn't have to do it, although it's certainly a great deal simpler than the previous best workaround, running TMG under Wine on a Linux box.

  John Cordes

[Jim Byram 0]

John,

Nice detective work.

Jim

[John Cordes 0]

Thanks Jim! I have just completed one more successful export. That makes 3 gedcom exports now, with no failures, with the IDrive service stopped. 

John