Susan J. 0 Report post Posted April 26, 2019 This week I discovered that my various LoW reports when run in my main database generate this error message: "... \appdata\local\temp\[alphanumeric string].tmp is too large" and I must exit TMG. I was able to run the reports in the sample project and in various client projects. Lee Hoffman was able to duplicate the error in his large project, but not in smaller projects. Regardless of project used, and regardless of output destination, LoW reports with subject ID # checked worked and LoW reports with subject "All witnesses" worked. All LoW reports with filtered subjects failed in my main database, but not in my smaller databases. Lee had the same results. At one point in my experimentation, I did get a FoxPro error message that began: "Fatal error: Exception code=C06D007E ..." and continued with a list of various "Called from" lines. I have run repair; I've restored the report defaults and created reports and filters from scratch with no changes in these results. I think this might be the result of the way memory is allocated in this report, but I don't know enough on that subject to make this conclusion. Details of my various experiments: On my main computer: TMG 9.05 under Windows 7; project has 18352 people and 81510 witnesses; first failed report had 9 columns, 610 character line length, output to Excel. When the "Create report" button is selected, the "Please wait" window comes up, then the hourglass, and finally the error message. Reducing column number and line length and changing output did not change the results. The same report, filter, etc. worked in the sample project and in my smaller projects. On my laptop, which is usually about 1 month behind my main computer in research progress: TMG 9.05 under Windows 10; project has 18122 people and 80504 witnesses. The report loads "5028 possible filter conditions" (something I don't see in my failed report), up pops the "Please wait" window, wait is about 2 minutes, and the filter window opens for field value input. I backed up my main computer database, omitting all filters, accents, etc., restored this to a new folder on my laptop, and tried to run a filtered LoW reports. The report failed. The slightly smaller March project still worked. My main project has two data sets in it, and the small set is disabled. Because I've been thinking about moving that set to its own project, I deleted it from the restored project on my laptop, and reran the filtered LoW report (still on laptop under Windows 10). This single-data set project has 18224 people and 81144 witnesses. The report worked! My main computer has more disk space and RAM than my laptop. What have I run into? Thank you for any suggestions. Susan Johnston Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted April 27, 2019 Jim, I think the tests run showing problems with larger data size may point to an issue with filter routines. I note that when TMG runs a filter, a window pops up which reports constructing all possible conditions. If these "all possible conditions" are being written to a Temp file, and the software library being used to match possibilities against that file is written as 16-bit functions, then the file could get too big for 16-bit functions to process against? Do you have any insight from the code as to whether filtering might use an old 16-bit library? Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted April 27, 2019 TMG9 shouldn't be using any 16-bit code. Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted April 28, 2019 Okay, Jim, any other ideas? Share this post Link to post Share on other sites
Jim Byram 0 Report post Posted April 29, 2019 I can output the report to Excel v5 with 290280 witnesses with no problem. In the spreadsheet with all witnesses, you should be able to do find what you want with sorting and searching as long as the output is properly structured. The use of a filter caused the report to end in an error. (I filtered for Tag Type... Label = BIRTH AND Country = AUSTRALIA. That gives a huge number of returns for this data set.) I tried several different output destinations. Text output failed much quicker than Excel v5 output. I haven't found any workaround. Share this post Link to post Share on other sites
Susan J. 0 Report post Posted April 29, 2019 Thank you, Michael and Jim, for working on this. Like you, Jim, I can create an Excel report listing all witnesses with no problems, so I have already started restructuring my currently filtered LoW reports so that I can filter, sort, etc., the list of all witnesses to get the result I want. As long as it's not a problem with my database, and I can use this workaround, I can live with it. Thank you again for testing this. Susan Johnston Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted April 30, 2019 Thanks. Jim, I will add a comment about this filter error to my "bugs" document, with your suggestion of using spreadsheet output as a work around. Share this post Link to post Share on other sites
Susan J. 0 Report post Posted April 30, 2019 Hi Michael, The workaround is more than simply using spreadsheet output. The use of any filter, regardless of output destination, generates the error if the number of witnesses in the project is large enough. I don't know what that "large enough" value is. The workaround involves generating a list of all witnesses to a spreadsheet and then sorting, selecting, filtering the resulting spreadsheet. This means that all filter options must be among the output columns in the spreadsheet report. I'm currently revamping my LoW reports with this in mind. To date, I've found only one problem. Although the LoW report filters allow filtering on any flag, the LoW report columns only allow standard flags. That does away with the utility of a Temporary flag for these LoW reports. I haven't used the Descendant/Ancestor Interest flags in many years, so I will probably treat one of them as a temporary flag for these reports. I can update you if I discover other speed bumps. Sue Johnston Share this post Link to post Share on other sites
Michael Hannah 0 Report post Posted April 30, 2019 Thanks, Susan, I did understand that the workaround was unfiltered spreadsheet output, i.e. all witnesses. However, I was unaware (or had forgotten) that the LOW output columns cannot show a non-standard Flag. Yes, please update me if you discover any other "speed bumps". See the two new entries in my Bugs document starting here: https://www.mjh-nm.net/BUGS.HTML#B103 Let me know if you have suggestions to improve the wording. Share this post Link to post Share on other sites
Susan J. 0 Report post Posted May 3, 2019 There is another speed bump in this workaround - the limitation in the number of rows in the Excel v. 5 output. A list of all witnesses is limited to 65,536 rows, so if the database is large enough to trigger the original "too large" error message, the number of witnesses is probably greater than the 65,536 rows. This limitation does not seem to exist in CSV output, so that output should be used in these "List of All Witnesses" reports. Any Excel version, 2007 and later, opens this report with no problems. Michael and Jim, do you find the same thing? Sue Johnston Share this post Link to post Share on other sites