Jump to content


Photo

TMG9 a Huge Disappointment


  • Please log in to reply
32 replies to this topic

#21 elevator

elevator
  • Senior Members
  • 147 posts
  • Gender:Male
  • Location:California, U.S.A.

Posted 20 April 2014 - 09:03 AM

These threads have indeed come up many times, but I certainly don't feel as though WhollyGenes have addressed the concerns; and I lurk both here and on the mailing list. We all know that TMG is built upon a dying platform and that a rewrite would, at some point, be necessary for this wonderful program to continue to compete with what's out there. I also think that most of us understand the cost associated with such an undertaking. Because of the work and cost involved I understand that a release for such an upgrade couldn't be promised; but personally I am not asking for that. All I want to know is that resources are being allocated to start this process. And as I said in a previous post; I'd be most willing to buy future upgrades (including this version 9) if I knew that resources from that sale would be allocated to move the database to a modern platform.

As for Unicode, there are many other genealogy programs that do not support it, but it is indeed a shrinking crowd. My concern is that as long as the very foundation upon which the program is built is outdated, fixing the Unicode problem will forever remain impossible.

Bottom line is; WhollyGenes could announce a long time goal of a platform update to ease the concerns of its user base without having to commit to an actual release date. In fact; this is the way they've been doing it all along. They have always been incredibly hesistant to reveal any information about their dedication to the development of the program. Updates appear sporadically and almost entirely without warning. No wonder long term users are worried about the long term viability of this wonderful program.

I wholehearedly share the views of the OP and some of the other posters here; TMG v9 was a disappointment. A minor upgrade without even as much as a pip about the future and the things that REALLY matter for the long term viability of the program. There is only one circumstance under which I will pay for mediocracy: if it comes with at least some roadmap for the future. In this case it is called an investment. Version 9 can, in my view, be called nothing else but mediocre. So if I am to "invest" in it I have no problem doing so if it comes with some indication that change is on the horizon.



#22 Helmut Leininger

Helmut Leininger
  • Senior Members
  • 263 posts
  • Gender:Male
  • Location:Vienna / Austria

Posted 20 April 2014 - 10:11 AM

Hi,

 

As for Unicode, there are many other genealogy programs that do not support it, but it is indeed a shrinking crowd. My concern is that as long as the very foundation upon which the program is built is outdated, fixing the Unicode problem will forever remain impossible.

As long as TMG stays based on Visual Foxpro, the Unicode problem cannot be solved.
 When TMG is migrated to another database, Unicode will come in "automatically" because I don't know an up-to-date database now that does not support Unicode.


Regards
Helmut

#23 jak

jak
  • Members
  • 7 posts

Posted 22 April 2014 - 07:46 AM

This topic has been around a while and has generated a variety of comments. I am a retired IT person whose knowledge is significantly out of date. I may be wrong in some of my comments and would be happy to be educated. That said, I think several of my original concerns still have some validity.

 

1. UNICODE

 

No genealogy program can be a professional product without support for non-Western alphabets. This seems to mean UNICODE support. Virtually every serious genealogist eventually links to ancestors from some part of the world which uses alphabetic characters not supported by TMG. This is completely unacceptable and should be an URGENT priority for Wholly Genes.

 

My impression is that the reason TMG does not support UNICODE is that it is based on MS Visual FoxPro, a long obsolete and no longer supported software environment. If this is correct, then UNICODE support would require a complete re-write of TMG using more modern software tools. I appreciate that this is a big job, but it is an issue that needs to be faced NOW.

 

The use of Visual FoxPro has many more implications. I suspect that VFP does not support multi-processors and large memory models, meaning that TMG does not make use of modern PC hardware, leading to unacceptably slow performance in a number of areas.

 

In addition, I suspect that the use of VFP means that TMG can not make use of current graphical interface features and must rely on older, less capable graphical features.

 

2. Database

 

My impression is that TMG does not use the SQL database or something similar. This leads to poor performance and makes TMG increasingly obsolete. Again, there is a performance issue - especially with large databases.

 

3. Data Interchange

 

TMG does not seem to offer high quality export of precious genealogical data to a neutral format for import into a competing product. To some extent, this appears to be a generic problem in genealogy and not solely a TMG problem. However, Wholly Genes should be at the forefront of developing and supporting a full featured neutral import/export format. This should be a priority for the entire genealogy community.

 

4. Cloud Support

 

Several members have commented on the cloud. In my view, I don't want my data on the cloud, I want it on my local PC. The cloud has some obvious advantages, but I only use cloud solutions for non-personal info and would never trust a cloud solution for personal info. Security is a critical issue for cloud based solutions.

 

 

The handwriting has been on the wall for several TMG versions that major changes are needed. It is the case that the above concerns have been raised previously on multiple occasions. I am not aware, however, that Wholly Genes has ever responded meaningfully to these concerns.

 

I am a huge fan of TMG and would like to continue with it. It has many powerful features. In my view it is one of the best genealogy programs on the market. That said, users have been waiting a long, long time for some indication from Wholly Genes that they are determined to upgrade their product to take advantage of modern hardware and software. We just can't continue down the path to obsolescence forever! At some point, many of us will be forced to jump ship - however painful that might be.



#24 Helmut Leininger

Helmut Leininger
  • Senior Members
  • 263 posts
  • Gender:Male
  • Location:Vienna / Austria

Posted 22 April 2014 - 09:59 AM

Hi Jak,

 

I have been working in IT for nearly 40 years, from old punched-card based systems to Unix (AIX) systems, supporting a lot of migrations, databases, building of High-Availabiity systems.

 

I agree with most of your statements, only a few comments from my point of view.

 

VFP does not support multi-processors and large memory models, meaning that TMG does not make use of modern PC hardware, leading to unacceptably slow performance in a number of areas.

I am not sure, if VFP can use multi-processors, although I have never seen TMG using more than 25% CPU on 4 4-processor machine. However, using more than one processor (at the same time) can only be done if several threads (tasks, works) can be run in parallel, i.e. are partially "independent" from each other and do not need the result from its companion. I have not investigated, if this would be possible in TMG.

 

My impression is that TMG does not use the SQL database or something similar. This leads to poor performance and makes TMG increasingly obsolete. Again, there is a performance issue - especially with large databases.

Microsoft describes VFP as an SQL-capable database. I do not know, if TMG uses the SQL-interface or a "classic" interface. Anyway, SQL itself does not give performance. It is "just" a standardised grammar for accessing relational databases and reduces the programming effort in the application. The performance increase (if any) comes from the database management - and most recent databases use large buffers to avoid disk accesses and have optimizers for recurring SELECT expressions.


Edited by Helmut Leininger, 22 April 2014 - 10:24 AM.

Regards
Helmut

#25 elevator

elevator
  • Senior Members
  • 147 posts
  • Gender:Male
  • Location:California, U.S.A.

Posted 23 April 2014 - 11:40 AM

This topic has been around a while and has generated a variety of comments. I am a retired IT person whose knowledge is significantly out of date. I may be wrong in some of my comments and would be happy to be educated. That said, I think several of my original concerns still have some validity.

 

[snip]

 

2. Database

 

My impression is that TMG does not use the SQL database or something similar. This leads to poor performance and makes TMG increasingly obsolete. Again, there is a performance issue - especially with large databases.

 

3. Data Interchange

 

TMG does not seem to offer high quality export of precious genealogical data to a neutral format for import into a competing product. To some extent, this appears to be a generic problem in genealogy and not solely a TMG problem. However, Wholly Genes should be at the forefront of developing and supporting a full featured neutral import/export format. This should be a priority for the entire genealogy community.

 

[snip]

 

I am wholeheartedly with you on the Unicode and the need get away from the VFP platform. However I disagree with point 3 and partly point 2; and here's why: the limitations on data interchange lies primarily with the GEDCOM standard, not with TMG. WhollyGenes is actually one of the few companies who (at least in the past) have published the database schema. Is this ideal? No. But the data composing family relationships and history can become quite complex and every software developer creates database schema's that fit their visions for the software. The problem is; unless each developer specifically implements import functions for competing database schemas, we're left with GEDCOM as the only data interchange standard for genealogical data. We cannot blame WhollyGenes or any other software developer for storing their data in different ways, and we cannot blame them for not imcluding import functions for every competing developer. Certainly, making the database 100% GEDCOM compatible would leave the database unable to deal with the complexities of modern genealogical databases.

Regarding point 2: this is a rather vague criticism. SQL isn't really a database system but rather a way to maniplulate data within a database. Most genealogy software today use some sort of proprietary format or they use a third party database engine. This engine may or may not be interfaced using SQL. In fact database engines may work in different ways; they may allow the user to issue direct SQL calls to the engine and get a dataset in return, or there may be preprogrammed procedural calls which when called from code accomplishes the same thing. And depending on what you mean by "SQL database", I would even say it could even be a security issue. Better in my view to abstract database calls into well-tested prcedural calls rather than exposing the database to standard SQL queries. One of the most serious security issues with web-based databases today are SQL injection attacks.

But like I said before; I largely agree with you: TMG must depart from the outdated VFP platform if it is to survive and with that departure UNICODE support will automatically follow. I just hope that despite the eerie silence from WhollyGenes such a rewrite is already on the drawing board.



#26 Helmut Leininger

Helmut Leininger
  • Senior Members
  • 263 posts
  • Gender:Male
  • Location:Vienna / Austria

Posted 23 April 2014 - 12:01 PM

Better in my view to abstract database calls into well-tested prcedural calls rather than exposing the database to standard SQL queries.

Only two remarks:

1) the mostly used databases today offer just an SQL interface, no "traditional" access

2) SQL-statements can be quite powerful and flexible and therefor reduce the programming effort (which also means costs) and testing of procedural calls quite al lot. It allows the programmer concentrate on the deployment of data, not their storage and retrieval.


Regards
Helmut

#27 elevator

elevator
  • Senior Members
  • 147 posts
  • Gender:Male
  • Location:California, U.S.A.

Posted 23 April 2014 - 12:34 PM

Only two remarks:

1) the mostly used databases today offer just an SQL interface, no "traditional" access

2) SQL-statements can be quite powerful and flexible and therefor reduce the programming effort (which also means costs) and testing of procedural calls quite al lot. It allows the programmer concentrate on the deployment of data, not their storage and retrieval.

 

I disagree Helmut; especially on your second point. So lets do the second point first; many database engines come with the procedural calls already developed. In fact if you were looking to "reduce the programming effort", as you say, it would be madness to develop a database engine from scratch in-house. Rather a third-party engine is used. Most such third-party solutions come with various ways of interfacing with database connections; be this SQL statements or procedural calls. Distributed applications (such as cloud environments) would benefit far more from a more secure approach than a desktop environment. This is common practice in the development world today; graphical engines are developed and used over a wide range of applications/games, same goes for audio libraries, physics libraries, etc., etc. Not only does such an approach dictate rapid application development but it also benefits from the testing already done on the existing framework.

On your first point; I'd say it is a mix. PHP for example, which is used for many web-applications, take a dual approach. They encapsulate the SQL calls in procedural calls so that the statements can be sanitized and evaluated for grammatical errors and injecton-type attacks. The way in which the calls are made internally I think is largely irrelevant. The average user aren't going to care squat about how the data is extracted or otherwise manipluated. The only time it would matter is for third-party addon support and such.



#28 jak

jak
  • Members
  • 7 posts

Posted 23 April 2014 - 02:15 PM

Thanks to those who have replied. We've gotten off into the weeds a bit on technical hardware/software issues of no interest to genealogists. Below is an additional 2 cents worth.

 

Non-Western Alphabet Support for Names and Places

 

The most critical deficiency of TMG I can see is lack of support for non-Western alphabets. To me this means UNICODE. I'm sure Wholly Genes are well aware of this and would dearly love to include UNICODE support. I believe the problem is the software environment - MS Visual FoxPro. According to Wikipedia (Comparison of Relational Database Management Systems) the last release of VFP was 2005 and the last update in 2007. Microsoft has been trying to kill it for many years and no longer supports it. In particular, VFP is incompatible with recent MS Windows software libraries and MS software environments like .NET. Virtually all current software environments include UNICODE support by default.

 

So, the message is - migrate as soon as possible to a currently supported, full featured software environment, You will get UNICODE support automatically and be able to take advantage of the latest software tools - multi-processor, multi-threading, access to large memory models, latest software libraries for graphical interface, database access, latest most highly optimized code libraries, etc.

 

Database Engine

 

Yes, strictly speaking, Structured Query Language (SQL) originally referred to a query language only. However, it had implications for the underlying database engine. Today, in my mind anyway, SQL implies both the query language and the underlying database engine. Microsoft's most powerful and capable product is called MS SQL Server. Presumably, the underlying engine is some sort of relational database. The point is that MS SQL Server (various versions) is far more capable and powerful than VFP. No serious application would use VFP today in preference to MS SQL Server. All of Microsoft's development effort is targeted for SQL. It supports large databases, parallelism (multi-threading), multi-processors and it has been and continues to be optimized in various ways for maximum power and speed, as well as for rapid development. There are alternatives like Oracle, Postgres, and others, but I would stick with Microsoft which offers a rich software environment for development.

 

A currently supported full featured database (whichever you choose) will blow the doors off VFP).

 

Data Interchange

 

I certainly agree that an interchange format is not just a Wholly Genes issue. But let me point out that GEDCOM was developed largely or entirely by the LDS church. The lack of a more powerful and capable GEDCOM replacement afflicts every genealogy program and company in the market. This does not let Wholly Genes off the hook.

 

A modern replacement for GEDCOM is urgently needed. Every genealogical software company that claims to be a leader in the market is a prime candidate for development of the replacement. If you are a market leader - develop and implement a viable GEDCOM replacement. There are a number of aborted XML based solutions out there. No one has stepped up as the LDS church did to solve this problem. The longer this problem festers, the harder it will be.

 

My goal here is just to encourage Wholly Genes to face the inevitable now. Move to a modern, supported environment and the potential for product improvement is enormous.



#29 RobinL

RobinL
  • Senior Members
  • 1,356 posts
  • Gender:Male
  • Location:Adelaide, Australia
  • Interests:charting, managing cultural differences in genealogical information, analysis for community and one-name studies

Posted 23 April 2014 - 03:11 PM

jak wrote:

Database Engine

 

.... MS SQL Server. All of Microsoft's development effort is targeted for SQL. It supports large databases, parallelism (multi-threading), multi-processors and it has been and continues to be optimized in various ways for maximum power and speed, as well as for rapid development. There are alternatives like Oracle, Postgres, and others, but I would stick with Microsoft which offers a rich software environment for development.

 

I believe that the major problem in moving to a more powerful database engine like one of those suggested is the IMPACT ON THE COST of the genealogical application. Unless Wholly Genes can use a FSF open-source database like Sqlite, it is probable that the individual licence cost would double or triple - making TMG non-competitive in the lower end marketplace. BTW: Sqlite is used in many common free applications like Mozilla Thunderbird and Firefox. UNICODE support is extremely important, but escaping from the obsolete VFP is even more important. That escape should automatically provide UNICODE support..

 

As far as data interchange is concerned, a rigid definition for interchange will never be satisfactory as the needs of genalogists and historian's continue to evolve (e.g DNA testing). XML is the best extensible vehicle for this data transfer in the future. Some XML examples have been explored, but there is a general lack of will in the family history software industry to provide this type of data interchange (as it would diminish the customer lock-in to a company's product).

 

RobinL


Robin Lamacraft
Adelaide, Australia

#30 Helmut Leininger

Helmut Leininger
  • Senior Members
  • 263 posts
  • Gender:Male
  • Location:Vienna / Austria

Posted 23 April 2014 - 10:23 PM

elevator wrote:

Most such third-party solutions come with various ways of interfacing with database connections; be this SQL statements or procedural calls.

Which ones? Do they really use "procedural calls" or just hide SQL ? What is your definition of a "procedural call?

 

They encapsulate the SQL calls in procedural calls

Then it is SQL.

 

The way in which the calls are made internally I think is largely irrelevant.

This contradicts what you said before:

One of the most serious security issues with web-based databases today are SQL injection attacks.

However, this applies much less to local databases.

 

RobinL wrote:

I believe that the major problem in moving to a more powerful database engine like one of those suggested is the IMPACT ON THE COST of the genealogical application.

I think this is a multi-part issue.

There are several databases out there that might be used without costs or with low cost only. But migrating the data from VFP to another database is the easiest part, is quite simple and could be done within a few days.

The problem is the application using the database (i.e. TMG itself). VFP includes also a development environment with powerful tools and a programming language. This must be replaced and this implies that most of the code has to be rewritten ...


Regards
Helmut

#31 elevator

elevator
  • Senior Members
  • 147 posts
  • Gender:Male
  • Location:California, U.S.A.

Posted 24 April 2014 - 12:09 PM

Which ones? Do they really use "procedural calls" or just hide SQL ? What is your definition of a "procedural call?

 

Of course if we use database systems like MySQL, Microsoft SQL Server, PostgreSQL, etc. then the interfacing with the database is necessarily done using SQL, but I think we're either talking past each other or misunderstanding what the other means by SQL calls compared to procedural calls. By procedural calls I mean the process of abstracting the database calls. In other words the application shouldn't be issuing SQL calls. The whole point of a third party library is to make procedural calls disassociated from the SQL syntax. This not only makes it easier to switch out database providers at will (as we see with the PEAR library in PHP, for example), but only makes it possible to sanitize any database call coming from the application. Sure, if you use a database system which inherently uses the SQL language to extract data then at the basic level all calls will be SQL calls, but this is misunderstanding how the system works. The whole point of "procedural calls" is to create an abstraction layer that could care less about the underlying technology. Does that clarify what I mean? As a point of comparison look, for example, at the GRAMPS project database, which currently uses Berkeley DB (in the past it was just XML).

 

 

Then it is SQL.

 

No. As I said above; abstraction and encapsulation are two different things. Abstraction makes different database technologies look like the same technology to the consuming application; encasulation does not.

 

 

This contradicts what you said before:

 

No it does not once you appreciate the difference. Operating systems are perfect examples. You can support hundreds of different types of hardware without the user application having to worry one bit about the underlying hardware technology. It is all dealt with by libraries further down the chain which creates layers of abstraction; essentially making every piece of hardware look identical to the consumer application. Imagine the headache if consumer applications had to implement raw hardware calls to every piece of hardware they wanted to support. Of course I am putting things a little on the edge here, but I hope you appreciate the difference.

 

 

However, this applies much less to local databases.

 

Of course, but cloud-based environments were one of the topics up for discussion. And I thought I made that quite clear in my post. Distributed databases are by definition more at risk than local databases are.



#32 Helmut Leininger

Helmut Leininger
  • Senior Members
  • 263 posts
  • Gender:Male
  • Location:Vienna / Austria

Posted 24 April 2014 - 09:59 PM

By procedural calls I mean the process of abstracting the database calls. In other words the application shouldn't be issuing SQL calls. The whole point of a third party library is to make procedural calls disassociated from the SQL syntax. This not only makes it easier to switch out database providers at will (as we see with the PEAR library in PHP, for example), but only makes it possible to sanitize any database call coming from the application.

Ok, it is clearer now. I thought that procedureal calls would mean traditional READ, WRITE, ... statements of "normal" files. That is clarified now. You are introducing a layer between the application and the database to have the same programming interface from the application whatever database is beyond. In my opinion, a good idea (it is relatively easy to exchange the database without influencing the application) when you do not use the "proprietory" DB SQL-extensions that every DB has (I don't think that TMG would have a need to do so).

But does it really make it less vulnerable to SQL injection attacks (although I don't think this being a big danger for TMG, and aslo non-SQL data may be attacked)?


Regards
Helmut

#33 elevator

elevator
  • Senior Members
  • 147 posts
  • Gender:Male
  • Location:California, U.S.A.

Posted 25 April 2014 - 07:42 PM

Yes, we're on the same page now, Helmut. Talking about attacks; I think pretty much any time information is put on a distributed system you have potential for attacks on this information. This may be going quite a bit off topic for this thread; but if we're talking about SQL injection attacks in particular, one major way to avoid it is by what Wikipedia calls "object-relational mapping" which keeps user input isolated from raw SQL calls. Injection attacks are, of course, only one way to compromise data in a database, but I assume other systems are vulnerable too; not only to injection attacks but also other attacks. On XQuery, for example, a simple Google search yielded many articles giving examples of not only injection type attacks, but many other types of attacks as well.



#34 cob666

cob666
  • Members
  • 11 posts
  • Gender:Male

Posted 01 July 2014 - 05:52 AM

I believe that the major problem in moving to a more powerful database engine like one of those suggested is the IMPACT ON THE COST of the genealogical application. Unless Wholly Genes can use a FSF open-source database like Sqlite, it is probable that the individual licence cost would double or triple - making TMG non-competitive in the lower end marketplace. BTW: Sqlite is used in many common free applications like Mozilla Thunderbird and Firefox. UNICODE support is extremely important, but escaping from the obsolete VFP is even more important. That escape should automatically provide UNICODE support..

 

RobinL

 

MS SQL Server Express is free so I don't see that replacing the database as a licensing cost issue.  

 

I've been in the IT field for a bit over 20 years and have worked at several companies with aging products.  The internal mentally is usually something like 'we use platform X because this is what we know, our product might not be as good if we switch over to Y'

 

Wholly Genes has a GREAT product but the lack of Unicode support is going to hurt TMG more and more, as long as they keep quiet about their roadmap.  It's only been pretty recent that Unicode has become an issue for me but there could be a time in the near future where Unicode support is the straw that breaks the camel's back.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users