Every eCommerce manager – working with product data or order processing – knows how tedious and time-consuming it is to efficiently manage these core aspects of an eCommerce store.
If you publish products one by one or do large imports, you are most likely wasting your time. Primarily if you work with many different suppliers.
The issue is mainly because suppliers use different platforms to distribute data, and they do so with very different structures of their files.
The same issue can be found in order fulfillment – whether you order to stock or ship directly from the supplier, you must manage many different ways of placing your orders. Forwarding emails, visiting B2B shops, or what’s worse.
This article will examine why improving these processes is essential and how you can do it.
It’s hard to manage manually with success
Three factors come into play:
- Speed of execution
- Costs
- Data freshness
Companies that automate will have a faster speed of execution, lower costs and more up-to-date data. The only way forward for those that don’t automate is to either move slower, have higher costs and/or settle with outdated product data.
Let’s dive into each section to understand the importance and impact better.
Speed of execution
The first question is: “Why is it important?”
It’s important because you’re in a market that revolves around your customers. Your competitors are also in that market.
If a brand launches a new product – or line of products – customers will more likely be interested in those products compared to the ones they launched 12 months ago.
Your competitors might have automated processes that allow them to start selling these new products within hours. This enables them to start marketing the products much faster than you and ultimately generate a lot more sales.
The same concept comes into play if you’re constantly taking in new brands or suppliers to grow your selection and sales. If each brand takes 3-6 months to onboard, you’re simply losing out on a lot of sales and marketing goodwill.
Final thoughts on this part are related to the other functions of your business. What is the most effective way for you to utilize your marketing department? And how do they collaborate with purchasing, merchandising, and warehousing?
Costs
Manual labor is expensive and prone to error.
The task of enriching and publishing products is not a very rewarding one either.
The math is quite simple. Let’s assume the average cost of employing a full-time employee to do this task is around €5,000/month. It could be a lot less or even more, depending on where you hire, the salary and benefits, and other overhead.
If they work 7 hours a day (effectively, which is very unlikely), and a product takes around 10 minutes each, they will enrich and publish 1680 products per month. With a cost of ~3 Euros per product.
If we also factor in that they have to retrieve, understand and locate the data they need, we can probably expect half, which is 840 products with a cost of almost 6 Euros each.
And if you have a month with no new products to publish, then you better find them some other tasks to do – you still have to pay their salary.
More savvy and experienced employees might use product bulk imports to save some time, but that brings us to our next topic.
Data freshness
How fresh is your data?
Well, even though it sounds silly, it’s a great way to understand this part of the subject.
If you published a product to your eCommerce store 3 months ago, you could have very fresh data or be stuck with moldy and useless data – all depending on your business and how you run things.
For instance, is it important to you what the current price of a product is, or how much is in stock in the brand’s warehouse?
If you only order in bulk and ship everything from your own warehouse, it might not be as important compared to an eCommerce store that relies more on dropshipping. (Read more about dropshipping here: https://stedger.com/what-is-dropshipping-pros-and-cons/)
In the case of carrying a large selection without having anything in stock, it’s extremely important to have fresh data, but almost impossible to manage manually – in this case, it’s a must-have to automate the process of live updating the data directly in your eCommerce platform.¨
What type of data and integrations do the suppliers offer?
Assuming you prefer to gain more speed in the market, lower your costs and gain a competitive advantage, we have to take a look at what’s possible.
It’s impossible to generate data out of nothing, so we’ll look at what suppliers and brands usually do to distribute their data and receive orders.
Since it’s split up into two parts, let’s start with the first part:
How suppliers share product data
Usually this is done with sheets of product data with columns such as SKU, description, image url (if you’re lucky), price and so forth.
Formats are typically:
- CSV
- Excel
- XML
The files are usually distributed through:
- HTTPS (a URL you can access)
- FTP
- API
There are many other ways, but you’d typically see some combination of the above-mentioned formats and protocols.
How suppliers receive orders
This is also done in different ways. Nowadays suppliers are becoming more digital which enables ordering through more than just the phone.
Typically we see:
- B2B store
- EDI
- API
You might find that some suppliers offer more than just one option, but either way they all require technical or manual work to get into your eCommerce platform or to place orders.
How can you automate product data flows?
Depending on what formats and protocols your suppliers offer, you have a few different options to automate the process – or at least make it easier.
We will go over:
- File imports
- Scraping
- Custom integrations
- No-code tools
- End-to-end integration with user interface
File imports
The first way to automate product data flows is through scheduled file imports.
There are tools for (probably) every eCommerce platform out there.
You can find most by simply going to Google and typing:
scheduled import insert eCommerce platform name
This should give you 5-10 different solutions you can try out and compare.
Here are some pros and cons when using file imports:
| Pros | Cons |
|---|---|
| Low price point | Need to remap if data structure changes |
| Not too technical | Need to setup each source one-by-one |
| Usually self-serve – get going quickly | Doesn’t necessarily support each supplier’s flow |
| Usually have good diagnostics tools | If images are not available, it’s not easy to import them as well |
| Using rules to modify data can be overwhelming |
Scraping
Imagine your supplier has no natural way of sharing your data, even though they have it all displayed on their website.
Let’s talk about scraping.
Web scraping refers to the automated process of extracting data from websites. It involves using software tools or programming scripts to gather information from web pages by sending HTTP requests, parsing the HTML content, and extracting specific data elements.
Typically, a web scraping process involves the following steps:
- Sending HTTP requests: The scraper sends requests to a target website’s server to retrieve the HTML content of the desired web pages. This can be done using programming libraries or tools.
- Parsing HTML: Once the HTML content is obtained, the scraper parses the markup language to understand its structure and extract the relevant data.
- Extracting data: The scraper identifies specific HTML elements (such as tags, classes, or IDs) that contain the desired data. It then extracts the data from these elements using techniques like CSS selectors or XPath expressions.
- Data processing and storage: After extracting the data, further processing and cleaning may be performed to ensure its quality and consistency. The scraped data can be saved in various formats, such as CSV, JSON, a database or potentially directly into your eCommerce platform.
It’s important to note that while web scraping itself is not illegal, the legality of scraping certain websites depends on the website’s terms of service and local laws.
When web scraping is used for gathering product data and images for eCommerce purposes, it is essential to consider certain aspects:
- Website permissions: Before scraping any website, it is crucial to review the website’s terms of service and check if they allow web scraping.
- Respect robots.txt: Websites often include a robots.txt file that specifies which parts of the website are off-limits to web crawlers and scrapers.
- Rate limiting and respectful scraping: Web scraping should be conducted in a responsible manner by implementing appropriate rate limiting. Excessive scraping can put a strain on the target website’s server and impact its performance.
- Copyright and intellectual property: When scraping product data and images, it is essential to respect copyright and intellectual property rights.
In general it is advised to be aware of and follow all legal and terms of service rules – preferably consulting a legal expert.
| Pros | Cons |
|---|---|
| Most websites can be scraped | If the layout is changed, the scraper will not work |
| If the supplier doesn’t provide any files, it still works | Can be tedious to setup |
| Can be automated | In most cases it’s quite technical to work with |
| The website owner might not want scrapers |
Custom integrations
This method has been used by large retailers for a long time. By approaching each supplier one-by-one, there’s usually a way to make a well-running integration with each one.
However, it is extremely technical, expensive, slow, and usually impossible to modify without software developers’ help.
The main issue with this approach is the fact that each integration is a one-to-one instead of one-to-many. It’s generally a lot better to simply integrate to one platform that can provide no-code rules, settings and aggregate all supplier data in one place.
Custom integrations are carried out either in-house or by outsourcing which also involves a lot of project management.
| Pros | Cons |
|---|---|
| Can work with almost any supplier | Very slow & time-consuming |
| The data can be shaped after any requirement | Very expensive |
| Usually reliable data quality | Needs maintenance |
| Requires software developers to make changes | |
| Integrations are done one-to-one – and one-by-one |
No-code tools
The past few years have been great for no-code. More and more companies are offering great solutions.
For instance, Make.com (formerly Integromat) have built an extensive library with different connectors and ways to modify data.
Another option is n8n.io and they’re both great.
Unfortunately, using these no-code tools sounds much better on paper than it really is. If you want to work with different formats, protocols, and structures, it will still be very technical and some might not even work with the available tools.
Furthermore there is always the risk of flows not really running which can have a negative impact on your whole setup.
Finally you also have to make changes everytime the structure of your supplier’s data changes.
| Pros | Cons |
|---|---|
| The basics are relatively simple | Still quite technical to make it work as intended |
| It’s quite visual | Might not have the necessary protocols or connections |
| Decent price-point | Flows can suddenly stop working |
| Many tutorials and communities to help you | Need to change every time a supplier changes their structure |
| Have to setup each one-by-one |
End-to-end integration with user interface
In my (slightly biased) opinion, this is by far the best solution.
You get the best of both worlds from no-code and custom integrations – easy to use but still very scalable, cheap, and customizable.
The most important aspect is that these platforms are built for this exact use case.
Full transparency – we’ve built such a platform because the existing options were so inadequate.
With a platform such as Stedger, you connect to the platform once, and voilà, you can simply click and publish products directly to your eCommerce platform from all of your connected suppliers.
How can you automate order management?
There are different levels of automation for orders that are all dependent on what types of orders your suppliers accept and what agreement you have with them.
Some are very simple to work with, whereas others are more demanding.
Automating your orders is almost a requirement in order to be extremely competitive and able to scale your business.
Steps of order management:
- Check for availability at the supplier.
- Split each order for each supplier.
- Place the order(s) – if you dropship, you will place a lot more orders with suppliers.
- Confirm that the order has been placed.
- Confirm that the order has been shipped.
- Receive track&trace for each order – if you dropship, this needs to be forwarded to your customers.
- Finalize the order and capture the payment.
We typically see orders places through emails, B2B shops and through integrations.
Here are the three ways you can start automating your orders:
- Custom integrations (API/EDI)
- Integrated order management solutions
Let’s learn more about each one of them.
If your suppliers accept orders by email, it’s very easy to automate.
Some eCommerce platforms have tools or apps that let you send an email each time an order has been placed. It is however a good idea to split the order, so that the supplier only receives an order for their specific line items.
This can be automated with tools like Zapier.com, N8n.io, or Make.com, but it does require some technical knowledge to do the proper setup.
If your volume increases a lot, a supplier might not want to receive orders through email if the flow is not automated on their end.
| Pros | Cons |
|---|---|
| Very fast to setup if a supplier accepts orders through email in a simple format | One-way communication |
| Cheap way to integrate | Can be a manual task on the supplier end, forcing them to shut it down |
| Hard to ensure the orders are split | |
| Rest of the order flow is still manual |
Custom integrations through API or EDI
Similarly to the section for custom integrations within product data, it is mainly carried out by large retailers as it’s very time-consuming and expensive to set up and maintain.
For many years, EDI has been the standard of B2B order integrations.
Working with EDI integrations requires a good understanding of the specific EDI standards and communication protocols, as well as collaboration with your B2B partners to align on message formats and requirements. It’s also important to consider the scalability and flexibility of your EDI solution to accommodate future changes or new trading partners.
API is a more modern standard that is widely used by most platforms.
APIs offer flexibility, real-time data exchange, and customization possibilities, making them well-suited for various integration scenarios.
EDI, on the other hand, is commonly used in industries with established standards and is more suitable for repetitive and standardized transactions.
The choice between API and EDI depends on the specific requirements, industry standards, and integration needs of the business. In some cases, a hybrid approach utilizing both API and EDI may be adopted to leverage the strengths of each method.
Integrated order management solutions
Fully integrated solutions are much easier to work with.
You simply integrate your eCommerce platform – or order management system – once, and the work on your end is done.
If you’re using a very popular eCommerce platform such as Shopify, WooCommerce, Magento or Prestashop, the set up can be done in less than a minute – or a few hours at a maximum.
Furthermore, these solutions offer power features such as automatically sending back track&trace numbers to your customers, automatically splitting and fulfilling orders.
Conclusion
In conclusion, the increasing complexity and scale of eCommerce call for automation, not only for efficiency but also for consistency and accuracy. The use of automation in product publishing and order processing is no longer a luxury; it’s a necessity for those who aspire to thrive in the eCommerce industry.
With innovative SaaS platforms like Stedger, the transition to automated systems is no longer an overwhelming task. By leveraging these tools, businesses can simplify their operations, reduce errors, and focus on their core functions, such as enhancing customer experience and implementing effective marketing strategies.
As we move forward in this dynamic landscape, automation stands out as a vital component of the eCommerce ecosystem. It’s not merely about keeping up with the competition; it’s about staying ahead. Embracing automation in eCommerce processes today is a step towards securing a successful and resilient future in the world of online retail.