The CRM import problem

Many companies still do manual imports, usually a file or files come in and believe me, they can come in from everywhere. At larger enterprise companies, there's a team or individual who is the one that gets sent these files to import. At smaller companies, this is usually a sales operations or CRM administrator who handles this.

These teams or individuals are treated like the most powerful people at a company. I've seen c-suite bow down to these Salesforce admins or sales operations individuals in the hope that the data will be imported quicker. I've seen sales managers tell their reps who are asking for more data to go outbound to, that they will have to wait until Dave (not a real person), the importer, decides their data is worthy of importing.

Why does this happen? To break it down into two perspectives, and I have been on both sides of it, we will look at the importer role first.

Importing data into a CRM is tricky business, especially with the complexity around CRMs now and all the different conditions that come with them. You have to make sure the data you are working with is all organized and formatted correctly, complex field mapping across tons of custom fields which are now available in a CRM, every record assigned to the right owner, and make sure you are selecting the right options to not overwrite anything important already in the CRM. Then one field mapping is slightly off or the import fails, and you have to start over again from the beginning.

Dave, and we'll use him as an example, is usually an experienced operator. Dave knows CRMs and has likely been in that role for a number of years. Companies don't let individuals fresh out of university have the keys to their entire platform which runs the business.

Dave gets files sent in from marketing, sales, GTM divisions and anyone else that wants their data importing. This is where the challenge arises for Dave, and the bottleneck. Dave will likely have a backlog of files waiting to be imported and will usually prioritize which files go in based on when they were submitted, to keep things fair. Everyone in the company will be aware they need to get files across and will have an idea of roughly how long they have to wait until the data is in the CRM.

They do not behave the same either, and the differences bite hardest at import time. Salesforce's Data Import Wizard takes 50,000 records at a time, so a bigger file means splitting it up or moving to Data Loader, a separate desktop application. HubSpot imports every row as a brand new record if your file has no unique column it recognizes, so a list you know is clean quietly doubles. Pipedrive wants custom fields created before the import rather than during it, and only a global admin can reverse one, within 48 hours. There's loads of issues that can arise, and they are all things Dave has to hold in his head at once. Move between two CRMs and he holds both sets at once, which is its own subject in Risks of migrating your CRM.

Before the import can even happen though, Dave must actually look at the file. Most of the time files that are sent aren't clean, ready to be imported data; they are usually a mess which has come from a marketing event or from a data provider. Whose job it was to clean that file before it reached Dave is a question most businesses have never settled, and Cleaning data: what it means, and who ends up doing it is about what happens when nobody does.

This is where Dave has a decision: import as much as possible and spend a few hours cleaning up the file to capture more data, or import what's possible in its current form. Either way, Dave gets a bashing from the reps who claim their data is bad, or loses hours on one file. With a backlog of files to go in already, most of the time it's a balance for Dave on capturing as much as possible without spending hours cleaning the file, and you can forget verifying it.

Now from the other perspective of those sending files to Dave. Because a lot of these have never even seen an import page on a CRM, they have no idea the magic Dave does to get the data that's on a spreadsheet in front of their faces ready to go in the CRM. This means that they have no idea what would make Dave's life easier and what they could do before sending the file.

I have seen companies implement an education process whereby any files being sent to Dave have to be in a very specific format and have the exact columns in the exact right order for it to be imported. While this does solve the issue marginally, suddenly every person sending a file must constrain what could be a rich data list into columns which may not capture all that they wanted. While the education piece works and makes Dave's life easier, it still ends in the same result, which is a loss of data.

1. 01Have a team of people doing the imports, screening and cleaning the files before they are imported. This works at mass volume and when the budget is there. However, it usually means giving extra responsibilities to sales ops, RevOps or CRM admins and adding to their workload.
2. 02Automate the system where data links directly from one place straight to the CRM through an API or similar. This is usually the solution many companies adopt when going through any digital transformation. Suddenly, you can get that data from a provider straight into your CRM and cut Dave out of the equation (sorry Dave).
3. 03Use AI to do the import process for you. This is like the more SaaS based automation but takes it a step above automating the system directly, whereby you use AI workflows to adapt the data coming in and then it gets imported to the CRM. This is also what a lot of the experts are doing now, using AI transformation on data as part of the automation project.

The second one does present the same issues. When it comes to systems mapping data to each other, systems aren't really set up to send and receive data in the right formats. So what you end up with is usually a worse result, where you click one button on one system, let's say a data provider, and then only half of that gets into the CRM and it turns out it's overwritten an enterprise account's contact and taken all of the notes off it.

This is where the experts usually come in to do a data quality audit and make sure the systems are speaking to each other in the right way. It does work. It also takes months, it isn't cheap, and at the end of it you've handed the keys over to those experts, who charge to maintain it and adapt it when you need them to. Should you automate your lead flow? goes through that trade with the figures.

The third one can work extremely well, where you have a system that adapts to different file types and levels of information and can even enrich that data before it enters the CRM, a job The hidden gap in migrating your CRM data covers in full. However, it presents a new list of challenges which we haven't covered before.

First, while AI has become increasingly advanced over the years, it still struggles with spreadsheets. It has a habit of inventing data, or saying it has done something when it hasn't. The reason it struggles is because of the volume of data there is and can be on a spreadsheet. AI can work well on a lower volume spreadsheet and do it exactly as you want, however ramp up the volume and this is where it struggles and starts inventing values or only doing half a job.

Second is the cost. Parsing a large spreadsheet uses tokens. The AI has to read and understand each value and what everything means before it can do the transformation, then actually do the transformation, then send it on its way. Tokens aren't cheap at volume, and this is how companies get saddled with a huge usage bill they weren't expecting.

Third, where your data actually goes. Handing your incoming data to an AI to transform and sort it means a third party is processing your customers' personal data, which makes them a sub-processor. That's a question about their contract rather than about the model: what they keep, for how long, where it sits, and whether any of it gets used to improve the service. Ask, and get the answer in writing. None of this is legal advice and your own agreement with them is what settles it.

The question to ask of anything in this category, ours included, is what actually leaves your workspace, and whether they can show you rather than tell you.

It's by combining the three together. Morphing Dave, the automated SaaS system and AI together so that each one has a role to play and doesn't overstep.

This is what Operelio does. We have Dave (the tools and importer master), the automated system (Trails) and the AI (Bridgeant) all working together to solve the issue. Let me explain how.

Dave is the toolkit

Dave is the one who knows the CRM importer process and is probably a wizard with spreadsheets. Operelio has 17 tools which turn any user into a spreadsheet wizard. That covers the cleaning, like merging the files, fixing the headers and removing the duplicates, and the verifying, like checking the email addresses and whether the company website is still live. It can all be done manually, from file to import, native in the system should a user want to.

All the tools themselves don't use AI, which means transformations are quick and accurate with nothing made up, and at a much lower cost.

The CRM Formatter tool, once set up with OAuth, will allow a user to map against their fields in their CRM, with a report at the end showing them exactly what will happen in their CRM before they do the actual import. This is covering things like the overwriting of data, the importing of duplicates or importing bad data. Native importers tell you what went wrong afterwards, in an error file, once the rows are already in. This tells you beforehand, and nothing reaches your CRM until you confirm it.

Trails is the automation

Trails allows you to set up folders which you can route certain files to, either by email, by uploading directly in the system, or using our API to import the data from another system directly. It cuts out that middle step of receiving all the different files coming in from everywhere, and gets them into one place.

You can set up a trail to schedule imports. So for example, a client sending you 7 files all across the week but you want them to enter the CRM at the same time: you schedule them all to go in on one trail, Monday morning the next week, already cleaned and verified and routed to the right users. The transformations on the files, such as deduplicating and verifying emails, are all done as part of a trail. You set what you want the trail to do, you test it, and once you are happy with it, it does the work within the parameters you set.

Bridgeant is the AI

Bridgeant is an AI agent with all the Operelio tools built in. The reason it's different from your standard AI chatbot is because it does the transformation using the tools themselves rather than reading every single value in your file.

It does three jobs and only one of them needs a trail. It advises, which is where it reads a file, scores it and tells you what is actually wrong with it and which route fixes it. It operates, which is where it plans the cleanup itself and runs the tools for you, once you have seen the plan and confirmed it. And it reviews, which is the one that needs a trail.

There are two things it will not do on its own, and both are deliberate. It will not spend your verification credits, so email verification stays a tool you open yourself once you have seen what it costs. And outside a trail it prepares the push rather than sending it, so the last click is yours.

Inside a trail it does none of that cleaning. It is added as a step just before the import to the CRM, and you give it instructions in your own words on what to block or let through or flag. That is the job when nobody is watching the run, and it is the only place Bridgeant can stop something: it can add a hold, and there is no answer it can give that means push it anyway. It can also advise you while you are building the trail in the first place.

Two things leave your workspace when Bridgeant reviews a push. The first is a payload of counts, column names and CRM field names, with no field in it that can hold a cell value: the type has nowhere to put one, a check throws if any unexpected key appears, and a test keeps that list clear of anything carrying raw data. The second is the instruction you typed. If you write "hold this if there are rows for Acme Ltd", then Acme Ltd goes out with the question, on every run. That's your own text and it's the point of the feature, but it's part of an honest answer to what we send.

The reason all three work well together is we are taking the best parts of each system. Dave's level of experience and knowledge, the automation's consistency and reliability, and the AI reasoning to react, flag and adapt when things aren't going right.

Put them together and each one covers what the other two can't, in something repeatable and easy enough to use that it gets to the end goal. Solving the data import problem into CRMs.