By Beate Thomsen, Co-founder & Product Design - September 27, 2023
Hello, and welcome to today's session. I will walk you through the Rapidi Data Integration Solutions and what we can do when it comes to, data integration projects.
First of all, my name is Andreea and I want to tell you a bit about Rapidi and walk you through the configuration interface
Rapidi provides solutions for cloud-based data integration, replication, and migration solutions. They are cloud-based but we also offer hybrid solutions and on-premise (locally installed) services.
Of course, we use our technology to integrate or replicate, any system or database, but we do, focus on, integrations between Salesforce and the Microsoft Dynamics products.
When we talk about RAPIDI, we have to talk about iPaaS, which is an integration platform as a service. This means that we are able to provide, cloud services and we enable the development execution of integration flows. And we are able to connect, both cloud on-premise applications as well as services and any other type of data, for any combination and for any type of company.
Of course, as I mentioned, when we mention Rapidi, we have to talk about iPaaS and the fact that there's no staging of your data it is very easy, to be configured. You don't really need any programming experience or expertise and we can connect multiple systems and companies. We have this possibility within the platform. We can accept any type of data. It can be custom or standard, basically any type of app. Our subscriptions are all-inclusive. In the sense that we do offer packages that include everything that you need, for your integration to be completed, accordingly.
Now I would like to show you an example of how this works and next I will show you the Rapidi integration platform.
First of all, let's take a look at this example, which is a Quote to Cash example. For example, what you see here on the left would be the Salesforce object. And then on the right, you will see the ERP objects or, let's say, tables.
Now, for example, We have a lead being created and converted, and this triggers the creation of an account, contact and opportunity.
Accounts, contacts and opportunities are synced and then sent to the ERP system. An account will be sent to the ERP as a customer record, and then a contact will be a contact and the opportunity or the quote will be a sales order.
So, what you can notice here is that data can go in one direction, but also can can go back to the ERP. So this would be bidirectional transfers. So it's up to you as a client, how this has to be configured depending on your business processes.
And then, once we have customers contact sales orders in the ERP, then we can send that information back to Salesforce. Like, in this case, sales orders would be sent as open sales orders. And then when it comes to invoices and payments, they can also be sent back to Salesforce as sales history and then payment history. So all the information can be sent back and forth. and everything can be up to date as long as we have the right process in place.
When it comes to the MyRapidi configuration interface, we do have to talk a bit about the Rapidi terminology that we use.
First of all, we have Connections: Connection is a link between, two systems, databases, or any type of company or entity.
Transfers: Transfers, which consist of tables, fields, all the mappings and filters and data conversion that you have for your specific company.
Schedules: A schedule is a schedule with actually refers to the way you control your transfers. For example, how often they are supposed to run? And, that you know, when exactly they are supposed to run. So we are talking about frequency, timing, and everything that has to do with the way that they are being run on a daily basis, so to speak.
System Log: Then we have the system log, which gives you information about what happens on our server. Of course, it's important to mention that we do not store any type of data. We do not store the transferred data.
Templates: Templates include predefined business flows, and they have been created, in order to help you as our client on your data integration project. And that's because templates are pre-built. They match the criteria, on both sides of your CRM and ERP systems. With the pre-build templates we then already have all the standard fields already there, built, and ready to be used.
So if you as a client are just aiming to have a standard solution implemented, then you can go ahead and use the templates and start testing them by using your own data. So by using the templates, you do save a lot of time and resources, and, of course, money because then you don't need to have resources assigned, working on any development on your side or mappings or anything like that. So it saves time and adds value to the entire process.
You can benefit from the integration really early in the process. You don't need to have to wait for a couple of months to have the mapping completed and then, start testing it. So you can do that right away, and you can enjoy it and then start testing. Then you can spend more time on testing your own use cases in more time on making sure that your data is accurate.
Service: And then service is the actual engine that is running the entire integration. So it contains everything, you know, from what I mentioned above, connections, transfers, schedules, and other functionalities that we have, already available.
Now let's take a look at the Rapidi platform. So this is an example of a service.
Here, you would have you would see, let's say, the status of your service when it will expire and your edition. And then the service status It actually allows you to get the health check of your service, see, how it's going And if there are any issues or anything, you'll definitely see it here. And then other two, functionalities that we have is the option to restart the service. If you want to, if you have issues on your side locally or if you have to restart your server locally, For example, then you can also do it here to make sure that everything is connected correctly and then delay all schedules.
You can delay all schedules. For example, if you want to work on any type of, extra development, and then you don't interfere with your schedules. If you, I mean, if you have schedules running, of course, you need to delay them. So then you can do your work without any, issues.
If you don't have any schedules and you don't need to use this, type of functionality.
Next, I will take a look at Connections. So connections, like I said, this is the first step when it comes to working on your integration.
You need to set up your source and destination connections, and you need to make sure that they work accordingly.
In order in order to set up a connection, you have to choose the connection tab from the list here and click on "new connection".
Once you do that, you have to fill in all the information that is required. In my case, for example, if I just take a look at one connection, then this is these are the fields that you need to fill in. And one, this, field here, the way you authenticate, it actually, determines the other fields that you will be seeing here. So if you use basic token or, Azure AD.
This is the information that you usually fill in. Of course, when you set up any connection, you do have the Wiki page at hand. So if you see here for every section, you have the question mark, If you open it in a different tab, you will be able to see information about, you know, how to configure a connection, what you need to add there, and everything that has to do with that. So the Wiki page can be used at all times for any kind of activity, done here.
So this is very useful, in case you want to set up your connection by yourself.
Moving on, we have here, the connection details. And then once you have them com completely configured, you just save it. And then the next step that you have here is test. You have to test it and make sure that this connection is actually working. You click on test, and then, we should be get we should be getting a message saying that it has been successfully connected. If you get error messages, then you have to check all the information that you have filled in, and you have to make sure that you have added the right one.
The second step is to redesign. So redesign is actually allowing disconnection to read all the tables that you have available in your system. But it's really important to mention that those tables need to be published.
Those pages need to be published. So they need to be available for us to read them.
So, every time you publish, a page, in your, source system, make sure you redesign because we only read what is actually made available, not everything that you have on your site.
So that would be for connections.
And then if we take a look at transfers
Let's move to Transfers
So as you can see, you have the templates already here. Now here on top, you have, some filters that can help you to actually see or find the transfers that you are interested in. First of all, this is a filter, that you can, add here to see only specific groups.
A group is actually, a set of transfers that are supposed to run together. So for example, you can have the item group. The item group would contain transfers, let's say one transfer that would bring in the product for the items. And then the other one would bring in the, prices, let's say. So,by grouping them together, then you have the possibility to have a a a clear picture of what you currently have for that specific page.
And then when it comes to invoices, we have the same thing. So you have invoices. You want to have one transfer to send invoice headers and then the other one to send invoice, let's say, invoice lines. So they are basically part of the same group. So it's easier for you to organize them here.
So that being said, you can add a filter that filters on items or customers.
So here, for example, let's take a look at one of the transfers. The starting point for you would be the general section, and here, this is where you need to start with your transfer.
So when it comes to the to the general section, you need to make sure that you are adding the right information.
First of all, the quote, this is a naming convention. You need to add description, the status that you want for it. The source would be the source system.
The layout, the same thing, destination, destination layout would be the destination system, where you need to send the data to. And then source table, which is a table that we need to use and then, where it needs to go.
So let me show you the next section Link Storages and what they actually do. Link Storages this is a functionality that we have available for those clients that want to store the primary keys.
So, for example, you have multiple entities; you have one entity in Denmark, one in Spain, and then you want to store the primary keys, here. So it'll be the source and destination. In this case, you'll have a link storage, which is configured. So then you get to see that information here.
But in order to configure a link storage, you need to to set up your primary keys on the source side and then the primary keys on the destination side. And then what a system does once you run the transfer, then all the data will be populated here as well. So you get to see, which is the customer number in business central and which is the Salesforce account ID. So it's, it's actually matching them here. And it allows you to store this information and then it helps you to, gather all the information that is needed.
As I mentioned before, you do have the Wiki page. So here, you can see all the information regarding the link storages, and of course, and if this is something useful for your company, you can, you can definitely, use it. But of course, like I mentioned, your store data, you can retrieve it. You can use the link storage to retrieve data from it and use it as a reference.
So it can be anything. But yeah, I think this is something that can definitely be used for companies that have multiple, legal entities or multiple locations. And you want to keep track of all the data that you, you have, right now.
So that would be the link storages, and they can be used on a specific transfer if, you need to. Like, in this case on this specific transfer you can use specific tags. Now You'll see here that we have link storages and tags. They actually go hand in hand, because the link storage here, you'll see under the setup of this transfer it shows all the tags that we have configured.
Let me show you the tags, so this makes sense.
Okay. So the tags here. The Tags are actually parameters, that we use in order to transfer data. You can see it as a criteria that you want to set up, and this is something that is being used for those companies, like I said, that have multiple legal entities or let's say there's a, a company that has offices in Denmark, Spain, France, Germany. And you want to keep the data separate. So in this way, you would have a tag here, a tag value for every company. So you have one for Denmark, one for Spain, and then for the other companies as well.
What you do with this tag is that you add information about the company and you say, okay, the connection should be this one, for Denmark for example, then you have the Salesforce connection that we need to use, and this would be the link storages that we want to use.
So that's why I was saying Link Storages and Tags go hand in hand because on one hand, we have the Link Storage, which functions as a Lookup table, basically and then the Tags which, actually incorporate the Link Storage and then it adds more criteria to that.
For example, if you have multiple companies, then you can use the same transfer, just one transfer, and then with the Tags set up you can transfer data for to for example five companies at the same time. But for that to happen, you need to have set up the Link Storage and then you also would need the Tags. Under the Tags and Tag Values you will see how your transfers will run based on the tags that you have configured. Under the Tag Values section you see, in this example, what Connection, Company, which Salesforce instance, The Link Storage that we will use, and the last 3 column would actually be the parameters from the actual Transfer - customers and then orders and the description. Above in the Tags section you have a short description also of what the Tag Values mean and how they are being used. But if you only have just one company or legal entity then you don't need to use Link Storages or Tags. You can just use the transfer as it is. But in those situations, if you want to, make sure that you have data into in in one place. You can use them combined. And then this will allow you to actually be more efficient when it actually, comes to sending data.
So like I said, when it comes to link storages, you can use, anyone listed here in the dropdown Link Storage under the Transfer. However, what is really important is that when it comes to the source and the destination, you can also you can also use the Tags that you have set up. For example under the dropdown Source you can use this Tag listed here (TAG: COMPANY - CONN) which would actually transfer the data based on the tags that you have added. You have two different companies here, and then Rapidi will know, how to route that data and where to send it. So that's something that, can be configured, in those cases where you do have a multiple legal legal entities.
But if you don't really have that, then you just set up the, source and then that's it. You just send the data over without any issues. As a side note on on the right side here under the General section of this Transfer, you have all the functions and actions that can be performed. Please check out the wiki page for more explanation on each of them.
The Wiki is the go to place that, any client can use constantly and make sure that everything is understood.
So here, everything is being explained, what each, action does and how you need to use it. So the wiki page is extremely, good. So you have all the, all the information that you need for that.
Then we have the source control. This is here. This this is, the section where you control basically what you send. So you have a control on the source side and on the destination side. So you can set up a field that is meant to control, like, what is being sent such as timestamps, fields. Right? And then you know, what needs to be sent. And, of course, you can do the same on the destination, side.
Transfers: Table Links
Then table links here you you need to mention which are the primary keys. So on the source side and on the destination side. So, if you have a customer, transfer, then in your source system, you have a primary key for that, and the same thing happens for destination.
Then field list mapping here. So here you'll see all the fields that, are already part of of the template. So on the left side, we have the source. On the right side, we have the destination. So here, you just need to add the field that you want. So, for example, if you just, write number, it just gives you the, the field that is currently available in the store system. Right. And then the same thing happens for the destination. So it has to have an equivalent there. What what you see here, this is a formula. And we do have, quite a few formulas that can be used, for integration purposes.
Transfers: Field List Mapping Formula
And this is something that, can be used if there's no, one to one, match. So for example, If I'm sharing this here, this is, the list with all the formulas that you can use. And if the and then an explanation for, each one of them. Let's say, for example, we want to take a look at this one. Right? So this would be a lookup. Done, on the, link storage. Right? So you have an explanation and then examples how you can use it.
And then other formulas, let's say we have, for example, let's see DB lookup, which is very, frequent. Right? So you are able to do a lookup to look up a value from a specific field in a in another table. So, like, if you're trying to map a field that's not in your table, then you can do a lookup and get it from somewhere else. So here, again, you have, explanations on how to use it and what they mean. And of course, tips and examples of how they should look like. So even though you don't have a one to one match, and you need, a field out somewhere else, then you can do a, a look up, and you can, pull the data from us from a from a different table. That's something, so formulas are are really useful when, you definitely need to bring over data in a specific format, or you want to bring it from a specific table, which is not right, added here on on the on the transfer.
Add Field to Mapping
And of course, if you want to add new fields, you just click on, new, you add the field that you have in your source, system, and then the field from your destination. And, of course, useful here you have the browse table layout. You can see the layout, from the, source. So here, you give to see the customer card, and you can search for specific, data. Let's see. Let's say bill bill to customer number. Like, this is something that I was able to find. So it's in our it's it's in my source system so that is available so you can use it. It specifies the type. You can see it has to be a string and the size. So it's pretty, easy. So you can definitely, find your field. And, of course, if you want to add it directly, you can do so. If not, just browse the table layout, and you can try to find it there. If you want to deactivate a field, then you just have to click on this disable function, and then it disables, it creates out the entire line. And you can leave it there as it is. So, basically, you will disable this line, but it will still show up on the transfer, but it will it will not take it will not be taken into account when the transfer will run.
Right? So you'll say I want to set up schedule for, invoices. Right? So all the transfers within the invoice group will run, as part of the schedule.
Here you need to add a description, interval how often do you need to run it?
Priority: You can set up priorities and say, okay, this transfer is a high priority. It needs to run, first, every time, and then the other one should be low. Right? So then Rapidi will know which one is the more important one.
Then linked to schedules, we have logs here. So the logs will show you, so three sections. You have runs. What exactly we we, we did so far.
So you can see here, it's more of, let's say, a timestamp of, the activity is done. And then you have the logs of everything, every single thing that we have done on this, service. And then the data errors, this will, actually this will actually show up when you do have data errors. When your transfer is running and you have data errors, you'll see them them listed here.
You can add filters, and then see specific errors from specific days. You can say, okay, when it started, when it ended, here you can say, okay, all transfers or maybe I want to see a specific transfer.
You can filter on source. Destination, and then the last filter here will be able to filter on errors which were not resolved or the one that were resolved. Or just all of them. So you can do that here.
And then you can see the error messages that are being displayed. Of course, we have the Wiki page here. That, will explain everything about data errors, logs, and runs, and, will definitely help you to understand how to use it.
And of course, the last, topic for today would be, a RTI.
So RTI here is another option functionality so to speed that we have. So I'm sharing the weekly page so you can see all the information about that. So the RTI is more of a reference book. So it will actually, keep track of, when data was, was transferred. And, it will be able to to know, okay, the transfer finished, at this point in time yesterday. And then, next time the transfer runs, rapidity will now to actually, continue from where it stopped. So it's more of, you know, avoiding, you know, running transfers, you know, every time.
Without, actually going through only the the data that has been changed. So it avoids spending time on tasks that have been already done previously. So it kind of knows where the transfer stopped. And then next time, it will continue from there. So this is something that, it is, useful in certain cases, and this is a functionality that we have. And, of course, clients can use it.
But, yes, I think that would be more or less about Rapidi Data Integration Application and Platform.
As I mentioned, you do have here the question mark, which, allows you to go to the Wiki page and you can read everything about, you know, about the functionalities that we have and the steps that you need to go through. In order to complete your, integration.
I think that was it. Thank you so much for listening. I hope you enjoyed it. And, if you have questions, feel free to reach out to us and if you have not already subscribe to our Product Updates Blog to receive news and updates about new features and improvements done on the MyRapidi Configuration Interface.
Thank you so much, Andreea Anseni - Data Integration Consultant at RAPIDI.
Beate Thomsen, Co-founder & Product Design
Walkthrough of MyRapidi Configuration Interface
Update on how to enable Passkeys on MyRapidi
WIKI - Rapidi Documentation integrated into MyRapidi