Jim Byram 0 Report post Posted September 1, 2014 (edited) This topic is for discussion of the GEDCOM export locked topic. The GEDCOM export 'enhanced' option has been in development and testing for the last month. Your feedback will be appreciated. http://www.whollygenes.com/forums201/index.php?/topic/15842-v9040000-gedcom-export-new-option/ Edited September 1, 2014 by Jim Byram Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 1, 2014 First a typo, in the GEDCOM export wizard it is "Step 5" not "Srep 5".I think the following HELP statement is either incomplete or misleading: None of the rules regarding what data is exported to a GEDCOM file when this option is not selected are changed. In other words, for a TMG tag with two Principals to be exported as a GEDCOM family record, the two principals must be married or must be the parents of one of more children (or both). Unless this extension makes changes to the export of two-Principal tags, I think the following statement is more clear and accurate: Unless explicitly mentioned below, none of the conditions in effect from previous TMG versions regarding what data is exported to a GEDCOM file are changed whether or not this option is selected. For example, for a TMG tag with two Principals to be exported as a GEDCOM family record, the two principals still must be married or must be the parents of one of more children (or both). And TMG Tag Types set to export as non-existent GEDCOM tag types instead of as EVEN/TYPE will still not export. Item 1. Since the user could have set either of these to EVEN/TYPE, I suggest: 1. Address and Telephone tags will now be exported, where previously by default they were not. Item 3. I "hope" Name tags can now *not* be set to anything *but* NAME?I believe the comment about changing to 2 DATE is dependent upon specific programs? 3. When Name tags are exported, dates will be exported as: 2 _DATE date For some import programs, this may need to be edited to 2 DATE after export in order for the date data to be recognized as dates upon import. Item 5. What about Witness memos? Are they still lost? Based on the introductory comment I presume so, but I think it is worthwhile to explicitly say so: The rolename will be whatever rolename the other witness is using when exported. There is no extended provision to export the witness memo. Item 8. What does the commment about GEDCOM PROP imply? Does the export check for a tag type set to export as PROP? If that is true I suggest: TMG doesn't have a default Property tag type. However if a user has customized a tag type to export as the GEDCOM PROP tag type, this provision for lengthy memos also has been implemented for that tag. =========================Hope this is helpful, Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 1, 2014 Question: How comprehensive is the check for the special GEDCOM export tag types? Is the check based on the TMG tag type name, or on the GEDCOM export tag type name? For example, if a custom TMG tag type has the GEDCOM export selection set to NOTE will the date export? Or if some custom TMG tag type has the GEDCOM export selection set to ADDR or PHON, will they export as the specified custom GEDCOM tag type names? Or what about TMG tag types set to export as CAST, DSCR, EDUC, OCCU and RELI, will the lengthy memo enhancment work with them? Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 1, 2014 (edited) Hope this is helpful, Some was and I'll make some edits. The import changes are practical and are only related to increasing the success of importing into RootsMagic and Legacy. Thanks. Adding witness memos. Thanks for bringing that up. Witness memos are supported both by RootsMagic 6 and by Legacy 8. (thanks to Tom Holden for the RM information) Edited September 1, 2014 by Jim Byram Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 1, 2014 Question: How comprehensive is the check for the special GEDCOM export tag types? That needs to be tested. For example, if a custom TMG tag type has the GEDCOM export selection set to NOTE will the date export? Yes. Tried that with an Anecdote tag and the 2 _DATE was exported. However, this uncovered a bug as the 2_SDATE tag was duplicated. Wrote the developer. However, I'll suggest that any custom tag type should have its export option set to 1 EVEN 2 TYPE. That way, you will always get the data to another program as a custom event and not get hung up by the limitations of particular GEDCOM tag types such as NOTE. Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 1, 2014 I completely agree with the suggestion to set to EVEN/TYPE to ensure appropriate output. However, users will ask about custom TMG tag types set to export as one of the GEDCOM tag types affected by these enhancements. Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 1, 2014 You mention in HELP to set the GEDCOM line length to a maximum line length of "246". Is that TMG's maximum limit, or just your recommendation? My notes show GEDCOM defines the max as 248. Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 2, 2014 The maximum line length that you can set in the TMG export wizard is 246 characters. (The actual output might be 248 characters with the CRLF but you'd need to count to determine that.) Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 2, 2014 By the way, you added the comment and example about witness memo to the (omitted) post and not the locked topic post. I think you want it in both places? Or is the locked topic waiting for additional other edits? I am confused about the fourth recommendation in HELP: "You can select the ID numbers option and the TMG REF_ID number (the ID number that you see in TMG) will be exported as the REFN value." Selecting ID number is in Step 5 of the Export Wizard. Step 7 has a radio button for the Reference field such that one of these options must be selected. What happens for output if the ID number is selected when one of these options selected? Will the ID number be output if there is no value in the selected field, but the selected value is output instead of ID if there is a value? or what? (omitted) Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 2, 2014 (edited) By the way, you added the comment and example about witness memo to the (omitted) post and not the locked topic post. I think you want it in both places? Are you talking about what's in post #4 above? Also, you appear to be discussing something that shouldn't be discussed on this forum. For that reason, I edited your post. Selecting ID number is in Step 5 of the Export Wizard. Step 7 has a radio button for the Reference field such that one of these options must be selected. The options on pages 5 and 7 are independent of one another. If you select the ID option on page 5 and the REFN choice on page 7, two REFN tags will be exported. And that isn't a problem. This has nothing to do with the additions to GEDCOM export and is the way that it has worked at least since TMG5 and probably before.. I made the export suggestion since that's the only way to get your TMG ID# into RootsMagic via GEDCOM. (Tom is looking at fooling RM into thinking that the GEDCOM came from RM. That gives you the choice of using the INDI number as the RIN. However, that might have unwanted consequences.) Edited September 2, 2014 by Jim Byram Share this post Link to post Share on other sites
TomH 0 Report post Posted September 2, 2014 The GEDCOM INDI number exported from TMG corresponds with a number in the database table that I do not see in the TMG UI. The visible number has been exported as a REFN in the two projects I have looked at. The two numbers do not necessarily match. RootsMagic has the capability to "Preserve record numbers" offered as an option for GEDCOM from RootsMagic and perhaps others but not for GEDCOM from TMG, I.e., the GEDCOM 0 @I532@ INDI line would create a record in RootsMagic with the Record Number 532. A trick may be to change the GEDCOM HEAD info from TMG to RootsMagic in order to trick it into offering that option. The first of multiple REFN values for a person can be optionally displayed after their name in RootsMagic but it is not as convenient as the Record Number for searching. Make sure that the more useful REFN is written first. If the record numbers of TMG were highly visible and RootsMagic preserved record numbers on import from TMG, user convenience would be improved. Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 2, 2014 The GEDCOM INDI number exported from TMG corresponds with a number in the database table that I do not see in the TMG UI. Huh? The INDI numbers in TMG-exported GEDCOMs come from the REF_ID value which is the default visible ID number for individuals seen in a TMG project. (The displayed number can be switched to the Reference number or both can be displayed.) The real ID of individuals in a TMG project that is used for database operations is the hidden PER_NO that is never seen by users. Both of these come from the $-table. Sent an email with screenshots and some details. Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 3, 2014 Tom mentioned: "the GEDCOM 0 @I532@ INDI line"I think you are misunderstanding INDI numbers, Tom. In nearly all export programs of which I have used including TMG, and as is expected and defined by the GEDCOM specifications, the INDI number in the GEDCOM file is intended to be simply the sequential number of that individual record in that file. That is, the individual record in the file is @I1@, the next record @I2@, and so on. These GEDCOM numbers are expected to have meaning only within this particular GEDCOM file. They are not expected to relate to any number internal to the exporting program or even be used by the importing program.This is the reason that GEDCOM defines the GEDCOM REFN tag. That tag is defined as "a user-defined number or text that the submitter uses to identify this record. For instance, it may be a record number within the submitter's automated or manual system..." The TMG Export Wizard provides the option to output the TMG ID number in an REFN tag. This is the number that is displayed on the screen for the individual, and which can be output in reports. Since that GEDCOM REFN tag is a standard tag, any importing program should retain this value associated with that individual. As for your suggestion to "Make sure that the more useful REFN is written first." GEDCOM specifies that neither the order of individual records in a GEDCOM file, nor the order of tag structures for a given individual, has any implied meaning. An importing program may choose to give precedence to an individual record or tag structure (or REFN) based on its order in the GEDCOM file, but that choice is arbitrary and not defined in the specifications. Hope this helps explain, Share this post Link to post Share on other sites
TomH 0 Report post Posted September 3, 2014 (edited) Thank you, Michael and Jim for the explanations. I do understand the role of the GEDCOM INDI record number, what I mistook was what TMG exported to it; I understand now that it is REF_ID and not the invisible PER_NO. RootsMagic, and I suspect some other programs, have the option to "preserve record numbers" on importing into an empty database, except that option is unavailable when importing a TMG GEDCOM. In RootsMagic, the unique Record Number is more convenient and effective for display, for user navigation and for searching than the non-singular REFN. It can optionally be displayed after the person's name and there is an accelerator key combo to find a person by RN. REFN can be an alternative post-name display (as can FamilySearch ID) but because there can be multiple REFNs for a person, only the first REFN record created is so displayed. Hence, the suggestion to output the REF_ID value as the first REFN, thinking that it would be the more useful and familiar if the INDI value (TMG REF_ID) could not be preserved in RootsMagic as its record number. I have since experimented and found a way to trick RootsMagic into offering that option - the last SOUR in the HEADer must have the value "RootsMagic". So it may be that some value from TMG other than the REF_ID could well be more advantageous as the first REFN but that may vary from one user to another. Certainly, importing into an existing database precludes the preservation of record numbers, in which case, it is likely that most would prefer the TMG REF_ID as the first REFN. My RMπ utility 1.0.3 now includes the aforementioned trick. Running a TMG GEDCOM through it and importing the processed GEDCOM into a new RootsMagic database with the "Preserve record numbers" option selected results in the RootsMagic Record Number (RIN) matching the TMG REF_ID. >edited 3 Sep 2014 14:20 UTC Edited September 3, 2014 by TomH Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 3, 2014 Thanks, Tom, for your utility. (I hadn't realized you were that Tom H. ) That should "larn" me not to teach grandma GEDCOM. Share this post Link to post Share on other sites
TomH 0 Report post Posted September 3, 2014 Oh, I'm larnin' from you guys. You'd be welcome additions to the RootsMagic user community. I think there's a deep understanding of GEDCOM and database systems in the TMG user community, viz the number and quality of utilities and addons of longstanding, the high level of much of the discussions in this forum and on TMG-L. Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 4, 2014 Why is this topic being hijacked for an academic discussion of GEDCOM standards that has nothing to do with this topic? Please move this discussion to a new topic. That said, the topic has attracted no user feedback on the new GEDCOM export feature other than Tom's posts which have led to changes in his RM post-processing utility. Tom continues to find new ways to improve the import to RM in ways beyond what can be done with the GEDCOM export process. Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 4, 2014 (edited) That issue is moot for two reasons. 1) It has nothing to do with the new GEDCOM export feature and isn't something that we can control in any event. 2) I've removed the ID Numbers suggestion from help. TMG exports the primary name tag first. That's what almost every program wants for import. I don't think that import by PAF is a real current concern. Edited September 4, 2014 by Jim Byram Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 4, 2014 Jim noted: "2) I've removed the ID Numbers suggestion from help." If I understand TomH's latest change to his utility, the presence of the TMG ID's can be of use to a RM import. I believe he indicates that with his utility RM could actually use the TMG ID's as its own ID number. Therefore I think the suggestion in HELP is of value. I would urge you to leave it in. I agree that the academic discussion here about the sequence of tags is off topic. Further, since the TMG export always puts the ID number REFN tag is the same place in the GEDCOM file, right after the 0 INDI tag, the sequence in a TMG export can be relied upon. Which makes this discussion evem more moot. Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 4, 2014 Therefore I think the suggestion in HELP is of value. I would urge you to leave it in. The suggestion was there for situations when the INDI could not be used as the RIN. That situation can be circumvented in RM with the post-processing and is not an issue with any other program that I've tested. Bottom line... the suggestion is no longer useful and just confuses things. The three help files already have been compiled and will be in the next build. Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 4, 2014 Hmmm... But I thought that IF the TMG ID was exported as an REFN, then the TMG ID could be used as the RM RIN instead of the INDI. Seems useful to me? Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted September 4, 2014 Again... The REF_ID is used as the INDI in TMG GEDCOM exports. Tom's post-processing lets the INDI be used as the RM RIN. So you have the same visible ID numbers being used in RM and in TMG. You could export the REF_ID as REFN tags using the ID Numbers option and display those next to the names in RM but why? You would have the same numbers as RINs and in Reference number facts. Other programs such as Legacy 8 give you the option to use the INDIs as the RINs. Family Historian uses the INDIs but I don't recall if that was an option or by default. I suspect that it was by default since FH uses the (updated) GEDCOM as it's database. The ID Number option can always be selected by the user for his/her export but I see no point in recommending it. Share this post Link to post Share on other sites
TomH 0 Report post Posted September 4, 2014 Some RM users wish for the ability to batch add a Reference Number to each person with the value of the current RIN. I wrote a SQLite query to do that and have used it on databases I have subsequently imported into my main database so that there was a reference back to the original. This is especially useful if there has been a book published from the original that made use of the record number. So I can understand that it may be desirable for the TMG reference number REF_ID to be exported both in INDI and in REFN, regardless of the destination database system. However, it may be desirable, as I have done, to prefix the number in the REFN with something that distinguishes it from other numbers, e.g., "TMG", which might then be readily searched and replaced by whatever the end user wants in some other database system. Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted September 5, 2014 Jim pointed out: "Again... The REF_ID is used as the INDI in TMG GEDCOM exports." Ahhh... that detail had escaped me. I had not noticed that feature in the TMG GEDCOM exports. All other programs I had tested did what GEDCOM expects, exports using consecutive INDI numbers with no gaps. Sorry about my confusion. Now I understand why the option to export the REF_ID as an additional REFN may have little meaning, especially if the importing program will use the INDI as its own RIN. Share this post Link to post Share on other sites
TPG 0 Report post Posted September 5, 2014 I am somewhat disquieted by the entire proposal to enhance the TMG gedcom export capability to accommodate the specific gedcom import facilities of other genealogy programs. I'm not sure why I am disturbed, since most of the tweeks added (e.g. the new initial underscore tags) would be ignored in a more general setting. But I do believe that it sets a bad precedent of junking up the standard TMG gedcom export capability with special features when perhaps cleaner options might exist. I am uncomfortable with the feature of TMG becoming dependent on the import conventions of other genealogy programs. In any case, the gedcom modifications proposed would seem to only approximate the natural import requirements of the specific importing program. I also ignores other competing products' import requirements. Rather, I would suggest that we rely on separate standalone programs to provide TMG output compatible with the import facilities of particular genealogy programs. These programs would, if necessary, read the TMG proprietary data files to produce the necessary input in whatever format was needed by the importing program. This is easily done as is shown by John Cardinal's TMG Utility and by the availability of the FoxPro OLE module underlying my WitnessTMG program. We could load up these individual standalone programs with as many features as desired and, if the associated genealogy program's import requirements change, only have to modify the individual program. TomH, while he was testing my witnessTMG program, urged me to adopt many of the suggestions he has made in this forum. I resisted incorporating the changes, partially on the grounds noted above. By the way, the modifications to the TMG export facility leaves intact what I consider to be the greatest deficiency in its gedcom export facility - namely the requirement that the owners of a joint event have to be married or at least be the joint parents of a child. Changing this to allow the event to appear as separate events under each individual should be relatively easy. Share this post Link to post Share on other sites