1

Suppose I have a code, where I run the scrapy crawler with python.

    process = CrawlerProcess({
        'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)'
    })

    data = str(self.prothom_date_lineedit.text())


    process.crawl(ArticleCommentsSpider, data)

    self.statusBar.showMessage("Scraping from prothom alo")

    process.start() # this starts the crawling. But I want to save the json file to a file.

As in command line, I always use scrapy crawl <proj_name> -o out.json.

How do I use this -o out.json in python?

Improve this question
2
Related questions
129
How to pass a user defined argument in scrapy spider
0
How to pass parameters or values to Python Scrapy Extension?
1
Python Scrapy: how to code the parameter instead of using cmd: use Custom code in Scrapy
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.