TMG & Zotero together

[Maccara 0]

Hi,

 

I have been using Zotero (www.zotero.org) for my other research work (simply excellent), and I have been wondering, if it would be possible to get some integration between Zotero & TMG to share bibliography data.

 

Is there a way to export source data from TMG in RIS or BibTeX format, for example (which I could then import to Zotero easily)?

I noticed the 'export sources' function outputs something with an extension '.xso', but it seems to be some binary format (which I did not recognize). Where could I find a description of this format? (I might be interested in developing a converter for this format, if one is not available already)

 

I especially like the Zotero Word AddIn and use it frequently (beats the Word 2007 built-in source/reference management system 10-0) as it allows me to use the same document for several publications by allowing me to change the citation style as required on the fly.

Edited October 27, 2008 by Maccara

[Virginia Blakelock 0]

I am also using Zotero - and had envisioned a direct integration between TMG and Zotero compatible sites. It is an elegant little program.

 

Virginia

[Michael Hannah 0]

Is there a way to export source data from TMG ...

I haven't used Zotero, but I browsed their web pages. For importing it seems that they like to start with web pages. Perhaps you might look at John Cardinal's TMG Utility program, and its "Make Source Page" feature. Perhaps you might consider developing a "translator" definition, as Zotero calls it, that would understand that web page format and import in to Zotero? Looks to me like it might be the simplest way to try first?

 

Hope this gives you ideas,

[Maccara 0]

I haven't used Zotero, but I browsed their web pages. For importing it seems that they like to start with web pages. Perhaps you might look at John Cardinal's TMG Utility program, and its "Make Source Page" feature. Perhaps you might consider developing a "translator" definition, as Zotero calls it, that would understand that web page format and import in to Zotero? Looks to me like it might be the simplest way to try first?

 

Zotero does better importing from straight RIS or BibTeX format for this purpose (they are designed for bibliography information, after all!), but that was an interesting idea.

 

Zotero translators for web-import is only a convenience feature and there are serious limitations what you can do that way (depends on meta-data available mostly).

 

(Zotero is not limited to web, it only has good convenience features for that! Especially for JSTOR et al direct collecting etc)

 

Checked what kind of output TMG Utility does, but that seems mostly useless for this purpose.

 

And, designing a "translator" (as Zotero calls them) for ambiguous web page source would be more work for me than developing a file converter for a well defined format from scratch! (that is, if the "xso" file is anything remotely well defined )

Edited October 28, 2008 by Maccara

[Maccara 0]

As a side note: I could also go "TMG Utility way" and access the DB directly, but I was envisioning more of a general solution which would allow me to easily do 2 way transfers (and maybe eventually some duplicate checking etc).

 

Direct write access to DB, in my view, is not a general solution (too risky, and I would have to check & double check compatibility for each new TMG version in case of schema changes etc) and that's why xso specs would be preferrable, in case there are any.

 

(well, extracting data is safe, so 1-way tool would be ok, even if it would break between TMG versions)

 

Of course, best would be, if TMG supported standard open formats already itself.

[Michael Hannah 0]

Checked what kind of output TMG Utility does, but that seems mostly useless for this purpose.

I forgot to mention it, but did you check the "Export Data" feature in TMG Utility? It also has the ability to export source information to an HTML page. It's pretty comprehensive, and includes more than the Make Source List feature. And the format is very fixed.

 

As for the .XSO and .XST file types used for exporting the sources and the source types respectively, they are standard files, in a way. I searched some old archives and found a mention that these files are actually standard ZIP files that are packed with the appropriate Visual FoxPro database files. If you simply open these zipped files with WinZip or a similar program (or rename the filetype to .zip to facilitate that) you will be able to work with the collection of files directly. As I understand it, they were zipped because it took multiple files to contain the information.

 

Hope this gives you ideas on how to proceed,

[Maccara 0]

Checked what kind of output TMG Utility does, but that seems mostly useless for this purpose.

I forgot to mention it, but did you check the "Export Data" feature in TMG Utility? It also has the ability to export source information to an HTML page. It's pretty comprehensive, and includes more than the Make Source List feature. And the format is very fixed.

 

Ah, didn't think of that! I'll check that out too.

 

As for the .XSO and .XST file types used for exporting the sources and the source types respectively, they are standard files, in a way. I searched some old archives and found a mention that these files are actually standard ZIP files that are packed with the appropriate Visual FoxPro database files.

 

Do'h! I should've looked in hex editor for that file myself. It indeed has a standard ZIP ID & header in the first bytes. (and of course opens just fine as a zip stream )

 

Hope this gives you ideas on how to proceed,

 

Most certainly, thank you very much.

 

Now that I know what those files contain and that I need to "reverse engineer" (checking schema or the html output you mentioned isn't really reverse engineering) anyway I have many options how to proceed.

 

I think Zotero was open source too (besides being free) so I could probably even implement this as a simple import/export functionality in Zotero for "xso" files - that would make a good general solution and available for everyone using Zotero (it already has custom import/export for EndNote, for example, so this kind of implementation wouldn't be out of scope and probably could be accepted as a contribution).

 

Only problem is, that I don't think VFP is multiplatform, so direct Zotero implementation might not be possible (or it might be difficult to get accepted). I'll check out if the html export output could be more feasible in this regard (but that won't allow 2-way import/export/merge).

 

So currently I'm leaning towards a stand-alone implementation (would allow more features, like 2-way merges of exported files, duplicate checking etc), but this may change.

 

Of course, there will be a few other issues to solve also (like, for example, how to handle custom source types etc - I don't believe Zotero is so customizable as TMG, and "output sentences" etc etc) but I think these are solvable.

 

The biggest problem for me will be how to find time to do this. Maybe Christmas vacation would be a good time slot to implement this.

 

If I come up with a nice & usable solution I'll release it also here for everyone to use (or if something is included in Zotero, I'll drop a note such a feature is/will be available in future Zotero versions).

 

Edit: of course, if someone has some ideas what would be "nice to have" functions in such integration/utility, feel free to drop me a note and I'll check if those would be feasible to implement.

Edited October 29, 2008 by Maccara

[Michael Hannah 0]

Only problem is, that I don't think VFP is multiplatform, so direct Zotero implementation might not be possible (or it might be difficult to get accepted). I'll check out if the html export output could be more feasible in this regard (but that won't allow 2-way import/export/merge).

 

So currently I'm leaning towards a stand-alone implementation (would allow more features, like 2-way merges of exported files, duplicate checking etc), but this may change.

Of course, TMG is not multiplatform, so the lack of a multiplatform implementation may not be such an issue. I think I agree that a separate standalone conversion utility would have the most flexibility. Using the .XSO and .XST files on the TMG side and some filetype on the other side that Zotero would both import and export might be more feasible and provide the 2-way import/export. TMG will automatically do the merge on import of .XSO/.XST file contents and my quick reading seems that Zotero will do merge of import on its side, so you may not need to focus on merge functions.

 

I am sure a number of us users would welcome this utility. Hope you find the time to work on it!

[Maccara 0]

Yes, I agree.

 

Seems most of the useful functionality would be fulfilled with just a simple "xso/xst <-> bibtex/ris" converter.

 

That should be simple to implement. If additional functionality is needed or nice to have it would be just a matter of adding it later.

Edited October 29, 2008 by Maccara