MyRapidi Log and Runs Optimized for Faster Response Time

By Michael Bock, Founder & CEO - April 03, 2015

In our ongoing effort to ensure fast response time of the MyRapidi platform we have done some changes to the Filtering and Search functionality under Log and Runs.

For MyRapidi Services that have Schedules that run very often, a lot of Log entries will be generated. These can often grow into millions of entries in just a few weeks or months.

We already have some cleaning and compression background jobs in place that helps us keep the Log tables in a reasonable size. You can read more about what is done in this blog post: MyRapidi Interface Runs and Statistics Optimized for Speed

We also have a quite flexible way to filter and search on which Log entries or Runs you will see. These possibilities include filtering on From and To Date, Schedule Code, Group Code, Transfer Code, IsError and also a generic search field for free-text search in the description and other fields. Read more on our wiki about Filtering and Search on MyRapidi 


But flexibility comes at a price and when there is a lot of Log entries, we need to ensure good performance by limiting the number of Log entries that are shown or searched through. Our primary way to limit the number of Log entries is our From and To Date filters. We already do some logic with counting the number of Log entries in order to show you a suitable selection of Log entries or Runs when you first click on the Log or Runs menu items - in general it will show you the last couple of days of Log entries, but if there is really a lot of Log entries, it might be reduced to the last couple of hours instead.

Checking on the number of Log Entries to show or search

You can then change the From and To Date to show other intervals and it's here that we have added some logic to ensure that we are not showing or searching too many Log entries. Now we check how many Log entries are in the interval that you have selected and if it's more than a certain number (currently set to 200.000), we issue a message to ask you to reduce the date interval selected. In this way, we allow as much flexibility as possible but without compromising the performance (too much).

Better indication of when data is retrieved on Logs and Runs

In order to give you more feedback on what is currently happening on the page, we now show our Spinning Wheel when the page is retrieving data from the server. This is true for both the Logs and Runs pages.

Updates to UI on the Statistics page

We have also updated the UI on the Statistics page. We show the Spinning wheel when data is retrieved and we have updated the styling of the menu buttons and in general, optimized this page for mobile devices also.

As always, feedback is very welcome


If you haven't already then subscribe to our "Product Updates" and stay up-to-date with new features and further development on MyRapidi.

Stay Tuned for more Product Updates  

About the author

Michael Bock, Founder & CEO

Picture of
Michael founded Rapidi on technological excellence, fantastic customer service and continuous improvement. A data integration specialist since 1987, he remains focused on creating technology that solves real business problems.