Moving Old Transactions To A Safe Store

inet01's Avatar

inet01

14 Jan, 2019 09:45 PM

I Have MD version 2017.7.1 (1671). Startup and shutdown is becoming quite slow, even on an SSD.It now has about 4 years worth of data. I would like to move the oldest two years of data from the main MD database to a safe store that can still be accessed by MD, should I need to look at those transactions. All the accounts are either bank or credit card, there are no investment accounts or anything like that.

Is this possible? If so, how? Does it calculate the correct opening balances for the accounts left in the main database?

  1. 1 Posted by mhoggie on 15 Jan, 2019 12:20 AM

    mhoggie's Avatar

    just a user,

    This is an issue that comes up often. At one time in the past MD did have an archive function to save and remove old year transactions, but there were issues with setting the correct starting balances on the current copy so that feature was removed. I have 8 years loaded an there are not issues. Others have decades loaded with no issues. But I understand your issue. Sometimes it would just be nice to offload the older past years if nothing else to reduce the listing when searching transactions. You can do this manually by just exporting a backup and using a descriptive name as needed (be sure to add the .moneydancearchive extension, MD does add the extension as needed on my win764 PC or some reason)
    Then once you have a backup, you can create a new file with a nice archive name, restore from the backup you made and delete anything you don't want on the backup archive.
    Then open your current active data set and delete transactions you kept on the backup and establish new starting balances so all your accounts have the correct balance again.

    Now you can see why MD chose to remove the yearly archive feature. It's would be a bit awkward to code because everyone has different needs. Especially involving investment transactions which some may want to never archive. Also remember that every transaction has two sides. If you want to keep all your investment and asset transactions, then deleting a checking transaction that transfers funds to investments would mess everything up. Yeah you can do it, but its a pain.

  2. 2 Posted by dwg on 15 Jan, 2019 12:56 AM

    dwg's Avatar

    I'm a fellow user.

    With Moneydance 2015 (When archive was removed) the data structures in Moneydance were significantly changes and the software moved to a journal approach to writing data, the upside of this is that it facilitates syncing, however it also means that any sort of archiving function actually increases the size of your data, since every addition, modification or deletion is represented by its own transaction.

  3. 3 Posted by inet01 on 15 Jan, 2019 10:55 AM

    inet01's Avatar

    Hi mhoggie,

    Thanks for getting back to me so quickly.

    The export procedure you describe sounds as though it has numerous points for something going wrong. Guess I'll just live with it.

    On 15/01/2019 00:20, mhoggie wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: mhoggie

    just a user,

    This is an issue that comes up often. At one time in the past MD did have an archive function to save and remove old year transactions, but there were issues with setting the correct starting balances on the current copy so that feature was removed. I have 8 years loaded an there are not issues. Others have decades loaded with no issues. But I understand your issue. Sometimes it would just be nice to offload the older past years if nothing else to reduce the listing when searching transactions. You can do this manually by just exporting a backup and using a descriptive name as needed (be sure to add the .moneydancearchive extension, MD does add the extension as needed on my win764 PC or some reason)
    Then once you have a backup, you can create a new file with a nice archive name, restore from the backup you made and delete anything you don't want on the backup archive.
    Then open your current active data set and delete transactions you kept on the backup and establish new starting balances so all your accounts have the correct balance again.

    Now you can see why MD chose to remove the yearly archive feature. It's would be a bit awkward to code because everyone has different needs. Especially involving investment transactions which some may want to never archive. Also remember that every transaction has two sides. If you want to keep all your investment and asset transactions, then deleting a checking transaction that transfers funds to investments would mess everything up. Yeah you can do it, but its a pain.

    On Mon, Jan 14 at 01:45 PM PST, inet01 wrote:

    I Have MD version 2017.7.1 (1671). Startup and shutdown is becoming quite slow, even on an SSD.It now has about 4 years worth of data. I would like to move the oldest two years of data from the main MD database to a safe store that can still be accessed by MD, should I need to look at those transactions. All the accounts are either bank or credit card, there are no investment accounts or anything like that.

    Is this possible? If so, how? Does it calculate the correct opening balances for the accounts left in the main database?

    Having trouble reading this? View this discussion online: Moving Old Transactions To A Safe Store.

    To unsubscribe and stop receiving emails from http://help.infinitekind.com, visit this page

  4. 4 Posted by inet01 on 15 Jan, 2019 10:56 AM

    inet01's Avatar

    Hi dwg,

    Thanks for getting back to me so quickly, and for the explanation of how it works. Guess I'll just have to live with it.

    On 15/01/2019 00:56, dwg wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: dwg

    I'm a fellow user.

    With Moneydance 2015 (When archive was removed) the data structures in Moneydance were significantly changes and the software moved to a journal approach to writing data, the upside of this is that it facilitates syncing, however it also means that any sort of archiving function actually increases the size of your data, since every addition, modification or deletion is represented by its own transaction.

    On Mon, Jan 14 at 04:20 PM PST, mhoggie wrote:

    just a user,

    This is an issue that comes up often. At one time in the past MD did have an archive function to save and remove old year transactions, but there were issues with setting the correct starting balances on the current copy so that feature was removed. I have 8 years loaded an there are not issues. Others have decades loaded with no issues. But I understand your issue. Sometimes it would just be nice to offload the older past years if nothing else to reduce the listing when searching transactions. You can do this manually by just exporting a backup and using a descriptive name as needed (be sure to add the .moneydancearchive extension, MD does add the extension as needed on my win764 PC or some reason)
    Then once you have a backup, you can create a new file with a nice archive name, restore from the backup you made and delete anything you don't want on the backup archive.
    Then open your current active data set and delete transactions you kept on the backup and establish new starting balances so all your accounts have the correct balance again.

    Now you can see why MD chose to remove the yearly archive feature. It's would be a bit awkward to code because everyone has different needs. Especially involving investment transactions which some may want to never archive. Also remember that every transaction has two sides. If you want to keep all your investment and asset transactions, then deleting a checking transaction that transfers funds to investments would mess everything up. Yeah you can do it, but its a pain.

    Having trouble reading this? View this discussion online: Moving Old Transactions To A Safe Store.

    To unsubscribe and stop receiving emails from http://help.infinitekind.com, visit this page

  5. Support Staff 5 Posted by Ethan on 15 Jan, 2019 02:43 PM

    Ethan's Avatar

    Hello,

    Everything others have written about archiving is correct. However, Moneydance should not be running as slow as you seem to describe, especially with just four years of non-investment data. Are you running any anti-virus, backup programs, or anything else that may be actively scanning your files? Sometimes it's necessary to exclude the Moneydance data files from these things when the program is in use, since they can slow down the program significantly.

    Can you send me your errlog.txt file? This should contain any error messages the program is generating, even if they're not being actively displayed. If you're using Mac OS X, it should be located at one of these locations within your Home folder:

    Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/errlog.txt

    or:

    Library/Application Support/Moneydance/errlog.txt

    If you have any trouble finding your Home Library folder, open Finder and select Go -> Go to Folder and type in:

    ~/Library

    This should pull up your local Library folder.

    If you're using Windows, it should be located at:

    C:/Users/(yourusername)/.moneydance/errlog.txt

    Ethan
    Moneydance Support

  6. 6 Posted by inet01 on 15 Jan, 2019 08:30 PM

    inet01's Avatar

    Hi Again, Ethan,

    Good to chat with you. The last errlog.txt file I could find is attached. However, I should explain a couple of things about my set-up, which is not a normal one:

    • it's a Windows set-up, as you will see from the log file, with MD installed in the normal place on the C: partition
    • the data files are stored in an encrypted file of the D: partition (single SATA SSD, with a C: partition and a D: partition, plus the MS system ones)
    • the encryption is Truecrypt

    There are multiple sets of MD data files in that encrypted file, but this one to which the errlog.txt file relates is the only active one.

    In addition, there are many other files in the encrypted file, for use with other applications, none of which have difficulty loading speedily. Although it is probably also true that none approaches the 7.5MB of the folder where all the MD data files resides.

    I use Kaspersky Internet Security for that side of things.

    Hope that helps.


    On 15/01/2019 14:43, Ethan wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: Ethan (Support staff)

    Hello,

    Everything others have written about archiving is correct. However, Moneydance should not be running as slow as you seem to describe, especially with just four years of non-investment data. Are you running any anti-virus, backup programs, or anything else that may be actively scanning your files? Sometimes it's necessary to exclude the Moneydance data files from these things when the program is in use, since they can slow down the program significantly.

    Can you send me your errlog.txt file? This should contain any error messages the program is generating, even if they're not being actively displayed. If you're using Mac OS X, it should be located at one of these locations within your Home folder:

    Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/errlog.txt

    or:

    Library/Application Support/Moneydance/errlog.txt

    If you have any trouble finding your Home Library folder, open Finder and select Go -> Go to Folder and type in:

    ~/Library

    This should pull up your local Library folder.

    If you're using Windows, it should be located at:

    C:/Users/(yourusername)/.moneydance/errlog.txt

    Ethan
    Moneydance Support

    On Tue, Jan 15 at 02:56 AM PST, inet01 wrote:

    Hi dwg,

    Thanks for getting back to me so quickly, and for the explanation of how it works. Guess I'll just have to live with it.

    On 15/01/2019 00:56, dwg wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: dwg

    I'm a fellow user.

    With Moneydance 2015 (When archive was removed) the data structures in Moneydance were significantly changes and the software moved to a journal approach to writing data, the upside of this is that it facilitates syncing, however it also means that any sort of archiving function actually increases the size of your data, since every addition, modification or deletion is represented by its own transaction.

    On Mon, Jan 14 at 04:20 PM PST, mhoggie wrote:

    just a user,

    This is an issue that comes up often. At one time in the past MD did have an archive function to save and remove old year transactions, but there were issues with setting the correct starting balances on the current copy so that feature was removed. I have 8 years loaded an there are not issues. Others have decades loaded with no issues. But I understand your issue. Sometimes it would just be nice to offload the older past years if nothing else to reduce the listing when searching transactions. You can do this manually by just exporting a backup and using a descriptive name as needed (be sure to add the .moneydancearchive extension, MD does add the extension as needed on my win764 PC or some reason)
    Then once you have a backup, you can create a new file with a nice archive name, restore from the backup you made and delete anything you don't want on the backup archive.
    Then open your current active data set and delete transactions you kept on the backup and establish new starting balances so all your accounts have the correct balance again.

    Now you can see why MD chose to remove the yearly archive feature. It's would be a bit awkward to code because everyone has different needs. Especially involving investment transactions which some may want to never archive. Also remember that every transaction has two sides. If you want to keep all your investment and asset transactions, then deleting a checking transaction that transfers funds to investments would mess everything up. Yeah you can do it, but its a pain.

    Having trouble reading this? View this discussion online: Moving Old Transactions To A Safe Store.

    To unsubscribe and stop receiving emails from http://help.infinitekind.com, visit

  7. Support Staff 7 Posted by Ethan on 15 Jan, 2019 09:09 PM

    Ethan's Avatar

    There's nothing obvious in the logs that would indicate Moneydance itself is taking a long time to do things. You are using a somewhat out of date version of Moneydance 2017, so you can try upgrading to the last release of that version from http://infinitekind.com/download_2017 if you want, but I'm not sure if that will affect this issue.

    Almost everything you mention in your setup could be a candidate for slowing down your experience. The Moneydance data file itself is encrypted by the program, so having that within a Truecrypt encryption could indeed slow things down. A few years ago we also had many cases where Kaspersky would slow down or prevent Moneydance from running at all. Usually these were solved by telling that program to not actively scan the Moneydance data file when the program was open, since Moneydance writes changes to that file every 20 seconds or so, and even more if you are syncing with other devices. But I haven't seen many of those cases since that glut of them a few years ago.

    This unfortunately may require some systematic testing on your end to get to the bottom of this, like disabling Kaspersky for a bit and seeing if that helps, moving your data file out of the encrypted Veracrypt file, different combinations of those, etc.

    Ethan
    Moneydance Support

  8. 8 Posted by david on 16 Jan, 2019 02:53 AM

    david's Avatar

    FYI, I've been using Moneydance for over 25 years, and I have only one performance irritation. Starting and interacting with Moneydance is no problem for me, but exiting is pretty slow (~20 seconds), as I've opted for "daily" backups at exit. The problem is that "daily" backups are actually created on every exit, but only one is kept each day. My guess is that speeding the backup isn't very feasible (though desirable), but it'd be nice if Moneydance didn't run a backup if a backup has already run that day.

  9. 9 Posted by dwg on 16 Jan, 2019 03:49 AM

    dwg's Avatar

    David,

    In the business world backups are usually done at the end of a day, given that it is impossible to know how many times moneydance will be run in a day running at each exit achieves the same goal. The ideal is to create the end of day backups then put them in a safe place. You can always run a backup manually instead.

    The backups in MD 2019 should be quicker. There is more compression and consolidation of the transaction files occurring in this version, so the data is smaller, plus Sean has put in a buffer for the use of Backup in this version.

  10. 10 Posted by david on 16 Jan, 2019 07:00 AM

    david's Avatar

    Thanks, dwg. I'm aware that most people would want their backups from the last session of the day; I was just asking for the option.

    That, of course, is moot if backups are faster. Good news about MD 2019.

  11. 11 Posted by inet01 on 16 Jan, 2019 12:39 PM

    inet01's Avatar

    Hi Ethan,

    Thanks for another swift response.

    MD was taking 27+sec to shutdown in the 2017 version. I had enquired about the 2019 version (upgrade costs) and had already downloaded it. I've just installed it. Startup time is marginally faster (maybe a sec or so), but shutdown is very much faster - about 4 sec.

    David, hope that is helpful for you - hopefully the backup time will be less of an issue..

    Thanks to all of you (dwg, David and Ethan) for your help on this, chalk it up as a success.

    On 15/01/2019 21:09, Ethan wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: Ethan (Support staff)

    There's nothing obvious in the logs that would indicate Moneydance itself is taking a long time to do things. You are using a somewhat out of date version of Moneydance 2017, so you can try upgrading to the last release of that version from http://infinitekind.com/download_2017 if you want, but I'm not sure if that will affect this issue.

    Almost everything you mention in your setup could be a candidate for slowing down your experience. The Moneydance data file itself is encrypted by the program, so having that within a Truecrypt encryption could indeed slow things down. A few years ago we also had many cases where Kaspersky would slow down or prevent Moneydance from running at all. Usually these were solved by telling that program to not actively scan the Moneydance data file when the program was open, since Moneydance writes changes to that file every 20 seconds or so, and even more if you are syncing with other devices. But I haven't seen many of those cases since that glut of them a few years ago.

    This unfortunately may require some systematic testing on your end to get to the bottom of this, like disabling Kaspersky for a bit and seeing if that helps, moving your data file out of the encrypted Veracrypt file, different combinations of those, etc.

    Ethan
    Moneydance Support

    On Tue, Jan 15 at 12:30 PM PST, inet01 wrote:

    Hi Again, Ethan,

    Good to chat with you. The last errlog.txt file I could find is attached. However, I should explain a couple of things about my set-up, which is not a normal one:

    • it's a Windows set-up, as you will see from the log file, with MD installed in the normal place on the C: partition
    • the data files are stored in an encrypted file of the D: partition (single SATA SSD, with a C: partition and a D: partition, plus the MS system ones)
    • the encryption is Truecrypt

    There are multiple sets of MD data files in that encrypted file, but this one to which the errlog.txt file relates is the only active one.

    In addition, there are many other files in the encrypted file, for use with other applications, none of which have difficulty loading speedily. Although it is probably also true that none approaches the 7.5MB of the folder where all the MD data files resides.

    I use Kaspersky Internet Security for that side of things.

    Hope that helps.


    On 15/01/2019 14:43, Ethan wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: Ethan (Support staff)

    Hello,

    Everything others have written about archiving is correct. However, Moneydance should not be running as slow as you seem to describe, especially with just four years of non-investment data. Are you running any anti-virus, backup programs, or anything else that may be actively scanning your files? Sometimes it's necessary to exclude the Moneydance data files from these things when the program is in use, since they can slow down the program significantly.

    Can you send me your errlog.txt file? This should contain any error messages the program is generating, even if they're not being actively displayed. If you're using Mac OS X, it should be located at one of these locations within your Home folder:

    Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/errlog.txt

    or:

    Library/Application Support/Moneydance/errlog.txt

    If you have any trouble finding your Home Library folder, open Finder and select Go -> Go to Folder and type in:

    ~/Library

    This should pull up your local Library folder.

    If you're using Windows, it should be located at:

    C:/Users/(yourusername)/.moneydance/errlog.txt

    Ethan
    Moneydance Support

    On Tue, Jan 15 at 02:56 AM PST, inet01 wrote:

    Hi dwg,

    Thanks for getting back to me so quickly, and for the explanation of how it works. Guess I'll just have to live with it.

    On 15/01/2019 00:56, dwg wrote:
    pre { width: 92%; margin: 10px 2%; padding: 5px 2%; background: #efefef; border: 1px solid #d6d6d6 } blockquote { margin-left: 0; padding-left: 1em; border-left: 5px solid #ccc; }

    // Please reply above this line
    ==================================================

    From: dwg

    I'm a fellow user.

    With Moneydance 2015 (When archive was removed) the data structures in Moneydance were significantly changes and (truncated)

  12. 12 Posted by david on 16 Jan, 2019 06:12 PM

    david's Avatar

    Very good news about MD 2019, inet01. Four seconds definitely makes this a non-problem.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac