The connector! An import/export story
Sooner or later you’ll get a request to create an import/export system between your application and one or more other applications. This is good: applications that are able to easily interact with the outside world will stand out and have more chance to be successful in the long run. So, you should happily grab the opportunity to make your application smarter. We do.
That said, to make it really smart you may want to think a little about the best way to do it. Or save time and just follow our advice, since we already spent some thoughts on it. The keyword here is: separation of concerns. You should build different components that work together to produce a perfectly working import/export system.
API layer
The first thing you should do is build a generic API layer, i.e. a standard way to read/write data from/to your application by another application. Our preferred choice is a REST webservice using JSON format for data and API keys for authentication, but that’s not strictly necessary. You may use CSV / XML file exchange, SOAP webservices, whatever you feel comfortable with.
Your main goal here is to make it generic. This layer should not know and care about other applications and their data structure. It should only know about your application data, and expose only those data and methods which are meant to be exposed.
Once your API layer is ready, your work may be already finished: other applications can connect via API following your excellent documentation. P.S. take a look to Swagger / OpenApi to make that easier.
Connector
Many times, you will also be in charge of retrieving data from other applications. Of course we will assume each of these applications have some way to expose their data; if not, you will have to repeat the API layer task for those applications as well – or ask some other developer to do that.
The second part of the system is what we call a connector, which is usually a separated Symfony CLI application, dedicated to a specific couple of applications. I repeat: separated. It is usually better to exclude this from your main application codebase, since there can be many different connectors.
The connector itself is made of three sub-components.
Connector: your API client
The connector should be able to reach for your application API layer, to read and write data. It should have an API client service, configurable with an URL and an API key, which knows all about the API layer rules. You built the API layer so building an API client for it should not be a problem. This sub-component does not know anything about the other system.
Connector: the other client
On the other end, the connector should be able to read and write data from the other system. Depending on what application you have to deal with, you may have to write a DB interface, a CSV reader, or another API client. This sub-component does not know anything except how to interact with the other system
Connector: the adapter
This is where the main business logic is: this sub-component knows how to transform data. It does not know anything about DB or REST or CSV or whatever: it only deals with arrays (or DTO) and performs all the transformations between the systems. It talks to both clients, asking them for data or giving them data to send.
Conclusions
Having a generic API layer and a separate connector will make it much easier to connect your application to different systems. Also, if you build the connector in the way we suggested, you may be able to reuse much of its code for the next connector. Reduce your concerns by separating concerns!