1

Is Importing a csv file through a shell script a good idea? Why yes and why not? What are the principal issues?

Thanks!

Improve this question

2 Answers 2

Reset to default
2

It depends how you approach it, I have on many occasions had no choice but to import CSV via a shell script - files coming from third party sources that cant be practical to split up.

Formats that are from non technical clients and not able to get them to put it into a format that the Magento Data importer works.

Writing a custom importer is relatively easy but you need to ensure you can handle error conditions in a sensible way. You also will need to ensure that all of the data attributes are pre-defined , or that you are able to create them on the during the import process.

Benefits

  • Import can work with any CSV (you would have to define the import script)
  • Fast Import can be achieved if bypassing Magento API and using SQL direct.
  • Can enable you to programatically generate your attributes/values on the fly.

Disadvantages

  • Requires Custom Logic
  • Risk of not importing all required information for proper operation of Magento (eg stock control)
  • No clear strategy for handling data - you need to define it properly.
  • By passing Magento API would require solid understanding of database structure.
  • Might break in future updates.
  • Updates from CSV might not be feasible - you might need to custom track between old and new ie if its a 3rd party and they delete the product how to identify that it should be deleted on next import.

There are some alternatives out there that I would reccomend you consider investigating before deciding that writing your own custom CSV importer is the correct strategy for your requirements. I have outlined two of the popular options below.

Magmi

Magmi is a mass importer for Magento, quite popular although it does have some bugs so it may not work for every senario. I've recently used it on a project and we had to disable some import functionality (to do with categories) because it broke price importing.

See More: http://sourceforge.net/projects/magmi/files/

API Import

It might not be using a CSV to do an import, but I think it might be worth considering. It was mentioned at a recent conference which I went to the author being a presenter. I think you will need to do a bit of pre import leg work to get it to the right setup but you can be fairly sure it will import correctly.

See More: https://github.com/danslo/ApiImport

Improve this answer
1
  • Thanks John! What would be a clear data handling strategy? What are the serious concerns to take into consideration? Thanks a lot! Commented Mar 7, 2015 at 18:49
2

I'm assuming you're talking about Magento shell scripts. You can achieve exactly the same functions with Magento shell scripts as you would with a proper extension if you just want to import CSV data. You use the same Magento methods.

I only see a few advantages of using Magento shell script including but not limited to:

  1. You can schedule it with the system cron, allowing you to avoid interrupting Magento cron jobs if the CSV import take a long time.

  2. No need to configure an extension.

The only time I use a shell script is when I'm migrating data (products, orders, etc.), or when there's a cron job that I know is intensive and takes up a lot of time, like bulk product creation/update, stock update, order syncing, etc.

With these kind of integrations, I usually create an extension that has all of the business logic and create shell scripts as entry points to the extension's processes. That way I can package my program properly and have it accessible by the system to be run independent of the Magento cron jobs.

Improve this answer
2
  • Do you mean that shell import is a better alternative for a cron job that's too intensive? Thanks a lot! Commented Mar 7, 2015 at 18:51
  • Yeah, I believe intensive Magento cron jobs configured in config.xml will hold up and cancel other queued cron jobs (depends system configuration) if it takes too long. I haven't verified in the latest versions (CE 1.9/EE 1.14), but it was the case previously. Commented Mar 8, 2015 at 5:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.