Jump to content
jak

TMG9 a Huge Disappointment

Recommended Posts

Better in my view to abstract database calls into well-tested prcedural calls rather than exposing the database to standard SQL queries.

Only two remarks:

1) the mostly used databases today offer just an SQL interface, no "traditional" access

2) SQL-statements can be quite powerful and flexible and therefor reduce the programming effort (which also means costs) and testing of procedural calls quite al lot. It allows the programmer concentrate on the deployment of data, not their storage and retrieval.

Share this post


Link to post
Share on other sites

Only two remarks:

1) the mostly used databases today offer just an SQL interface, no "traditional" access

2) SQL-statements can be quite powerful and flexible and therefor reduce the programming effort (which also means costs) and testing of procedural calls quite al lot. It allows the programmer concentrate on the deployment of data, not their storage and retrieval.

 

I disagree Helmut; especially on your second point. So lets do the second point first; many database engines come with the procedural calls already developed. In fact if you were looking to "reduce the programming effort", as you say, it would be madness to develop a database engine from scratch in-house. Rather a third-party engine is used. Most such third-party solutions come with various ways of interfacing with database connections; be this SQL statements or procedural calls. Distributed applications (such as cloud environments) would benefit far more from a more secure approach than a desktop environment. This is common practice in the development world today; graphical engines are developed and used over a wide range of applications/games, same goes for audio libraries, physics libraries, etc., etc. Not only does such an approach dictate rapid application development but it also benefits from the testing already done on the existing framework.

 

On your first point; I'd say it is a mix. PHP for example, which is used for many web-applications, take a dual approach. They encapsulate the SQL calls in procedural calls so that the statements can be sanitized and evaluated for grammatical errors and injecton-type attacks. The way in which the calls are made internally I think is largely irrelevant. The average user aren't going to care squat about how the data is extracted or otherwise manipluated. The only time it would matter is for third-party addon support and such.

Share this post


Link to post
Share on other sites

Thanks to those who have replied. We've gotten off into the weeds a bit on technical hardware/software issues of no interest to genealogists. Below is an additional 2 cents worth.

 

Non-Western Alphabet Support for Names and Places

 

The most critical deficiency of TMG I can see is lack of support for non-Western alphabets. To me this means UNICODE. I'm sure Wholly Genes are well aware of this and would dearly love to include UNICODE support. I believe the problem is the software environment - MS Visual FoxPro. According to Wikipedia (Comparison of Relational Database Management Systems) the last release of VFP was 2005 and the last update in 2007. Microsoft has been trying to kill it for many years and no longer supports it. In particular, VFP is incompatible with recent MS Windows software libraries and MS software environments like .NET. Virtually all current software environments include UNICODE support by default.

 

So, the message is - migrate as soon as possible to a currently supported, full featured software environment, You will get UNICODE support automatically and be able to take advantage of the latest software tools - multi-processor, multi-threading, access to large memory models, latest software libraries for graphical interface, database access, latest most highly optimized code libraries, etc.

 

Database Engine

 

Yes, strictly speaking, Structured Query Language (SQL) originally referred to a query language only. However, it had implications for the underlying database engine. Today, in my mind anyway, SQL implies both the query language and the underlying database engine. Microsoft's most powerful and capable product is called MS SQL Server. Presumably, the underlying engine is some sort of relational database. The point is that MS SQL Server (various versions) is far more capable and powerful than VFP. No serious application would use VFP today in preference to MS SQL Server. All of Microsoft's development effort is targeted for SQL. It supports large databases, parallelism (multi-threading), multi-processors and it has been and continues to be optimized in various ways for maximum power and speed, as well as for rapid development. There are alternatives like Oracle, Postgres, and others, but I would stick with Microsoft which offers a rich software environment for development.

 

A currently supported full featured database (whichever you choose) will blow the doors off VFP).

 

Data Interchange

 

I certainly agree that an interchange format is not just a Wholly Genes issue. But let me point out that GEDCOM was developed largely or entirely by the LDS church. The lack of a more powerful and capable GEDCOM replacement afflicts every genealogy program and company in the market. This does not let Wholly Genes off the hook.

 

A modern replacement for GEDCOM is urgently needed. Every genealogical software company that claims to be a leader in the market is a prime candidate for development of the replacement. If you are a market leader - develop and implement a viable GEDCOM replacement. There are a number of aborted XML based solutions out there. No one has stepped up as the LDS church did to solve this problem. The longer this problem festers, the harder it will be.

 

My goal here is just to encourage Wholly Genes to face the inevitable now. Move to a modern, supported environment and the potential for product improvement is enormous.

Share this post


Link to post
Share on other sites

jak wrote:

Database Engine

 

.... MS SQL Server. All of Microsoft's development effort is targeted for SQL. It supports large databases, parallelism (multi-threading), multi-processors and it has been and continues to be optimized in various ways for maximum power and speed, as well as for rapid development. There are alternatives like Oracle, Postgres, and others, but I would stick with Microsoft which offers a rich software environment for development.

 

I believe that the major problem in moving to a more powerful database engine like one of those suggested is the IMPACT ON THE COST of the genealogical application. Unless Wholly Genes can use a FSF open-source database like Sqlite, it is probable that the individual licence cost would double or triple - making TMG non-competitive in the lower end marketplace. BTW: Sqlite is used in many common free applications like Mozilla Thunderbird and Firefox. UNICODE support is extremely important, but escaping from the obsolete VFP is even more important. That escape should automatically provide UNICODE support..

 

As far as data interchange is concerned, a rigid definition for interchange will never be satisfactory as the needs of genalogists and historian's continue to evolve (e.g DNA testing). XML is the best extensible vehicle for this data transfer in the future. Some XML examples have been explored, but there is a general lack of will in the family history software industry to provide this type of data interchange (as it would diminish the customer lock-in to a company's product).

 

RobinL

Share this post


Link to post
Share on other sites

elevator wrote:

Most such third-party solutions come with various ways of interfacing with database connections; be this SQL statements or procedural calls.

Which ones? Do they really use "procedural calls" or just hide SQL ? What is your definition of a "procedural call?

 

They encapsulate the SQL calls in procedural calls

Then it is SQL.

 

The way in which the calls are made internally I think is largely irrelevant.

This contradicts what you said before:

One of the most serious security issues with web-based databases today are SQL injection attacks.

However, this applies much less to local databases.

 

RobinL wrote:

I believe that the major problem in moving to a more powerful database engine like one of those suggested is the IMPACT ON THE COST of the genealogical application.

I think this is a multi-part issue.

There are several databases out there that might be used without costs or with low cost only. But migrating the data from VFP to another database is the easiest part, is quite simple and could be done within a few days.

The problem is the application using the database (i.e. TMG itself). VFP includes also a development environment with powerful tools and a programming language. This must be replaced and this implies that most of the code has to be rewritten ...

Share this post


Link to post
Share on other sites

Which ones? Do they really use "procedural calls" or just hide SQL ? What is your definition of a "procedural call?

 

Of course if we use database systems like MySQL, Microsoft SQL Server, PostgreSQL, etc. then the interfacing with the database is necessarily done using SQL, but I think we're either talking past each other or misunderstanding what the other means by SQL calls compared to procedural calls. By procedural calls I mean the process of abstracting the database calls. In other words the application shouldn't be issuing SQL calls. The whole point of a third party library is to make procedural calls disassociated from the SQL syntax. This not only makes it easier to switch out database providers at will (as we see with the PEAR library in PHP, for example), but only makes it possible to sanitize any database call coming from the application. Sure, if you use a database system which inherently uses the SQL language to extract data then at the basic level all calls will be SQL calls, but this is misunderstanding how the system works. The whole point of "procedural calls" is to create an abstraction layer that could care less about the underlying technology. Does that clarify what I mean? As a point of comparison look, for example, at the GRAMPS project database, which currently uses Berkeley DB (in the past it was just XML).

 

 

Then it is SQL.

 

No. As I said above; abstraction and encapsulation are two different things. Abstraction makes different database technologies look like the same technology to the consuming application; encasulation does not.

 

 

This contradicts what you said before:

 

No it does not once you appreciate the difference. Operating systems are perfect examples. You can support hundreds of different types of hardware without the user application having to worry one bit about the underlying hardware technology. It is all dealt with by libraries further down the chain which creates layers of abstraction; essentially making every piece of hardware look identical to the consumer application. Imagine the headache if consumer applications had to implement raw hardware calls to every piece of hardware they wanted to support. Of course I am putting things a little on the edge here, but I hope you appreciate the difference.

 

 

However, this applies much less to local databases.

 

Of course, but cloud-based environments were one of the topics up for discussion. And I thought I made that quite clear in my post. Distributed databases are by definition more at risk than local databases are.

Share this post


Link to post
Share on other sites

By procedural calls I mean the process of abstracting the database calls. In other words the application shouldn't be issuing SQL calls. The whole point of a third party library is to make procedural calls disassociated from the SQL syntax. This not only makes it easier to switch out database providers at will (as we see with the PEAR library in PHP, for example), but only makes it possible to sanitize any database call coming from the application.

Ok, it is clearer now. I thought that procedureal calls would mean traditional READ, WRITE, ... statements of "normal" files. That is clarified now. You are introducing a layer between the application and the database to have the same programming interface from the application whatever database is beyond. In my opinion, a good idea (it is relatively easy to exchange the database without influencing the application) when you do not use the "proprietory" DB SQL-extensions that every DB has (I don't think that TMG would have a need to do so).

But does it really make it less vulnerable to SQL injection attacks (although I don't think this being a big danger for TMG, and aslo non-SQL data may be attacked)?

Share this post


Link to post
Share on other sites

Yes, we're on the same page now, Helmut. Talking about attacks; I think pretty much any time information is put on a distributed system you have potential for attacks on this information. This may be going quite a bit off topic for this thread; but if we're talking about SQL injection attacks in particular, one major way to avoid it is by what Wikipedia calls "object-relational mapping" which keeps user input isolated from raw SQL calls. Injection attacks are, of course, only one way to compromise data in a database, but I assume other systems are vulnerable too; not only to injection attacks but also other attacks. On XQuery, for example, a simple Google search yielded many articles giving examples of not only injection type attacks, but many other types of attacks as well.

Share this post


Link to post
Share on other sites

I believe that the major problem in moving to a more powerful database engine like one of those suggested is the IMPACT ON THE COST of the genealogical application. Unless Wholly Genes can use a FSF open-source database like Sqlite, it is probable that the individual licence cost would double or triple - making TMG non-competitive in the lower end marketplace. BTW: Sqlite is used in many common free applications like Mozilla Thunderbird and Firefox. UNICODE support is extremely important, but escaping from the obsolete VFP is even more important. That escape should automatically provide UNICODE support..

 

RobinL

 

MS SQL Server Express is free so I don't see that replacing the database as a licensing cost issue.

 

I've been in the IT field for a bit over 20 years and have worked at several companies with aging products. The internal mentally is usually something like 'we use platform X because this is what we know, our product might not be as good if we switch over to Y'

 

Wholly Genes has a GREAT product but the lack of Unicode support is going to hurt TMG more and more, as long as they keep quiet about their roadmap. It's only been pretty recent that Unicode has become an issue for me but there could be a time in the near future where Unicode support is the straw that breaks the camel's back.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×