web scraping - How to execute scrapy from php on Parallel? -

i have 90 urls in array

[url1,url2,url3,  ... , url90] 

i want have 3 spiders works @ same time , pass 1 url each one, so, first instance of scrapy gets url1, second 1 gets url2 , third 1 gets url3, , when first 1 finish job url4.

i used gnu parallel if there software better use that.

i tried 1 in php because should launch scrapy php

exec (for url in urlstab | parallel -j 3 scrapy crawl myspider -a url {})  

you want (untested):

$parallel = popen("parallel -j 3 scrapy crawl myspider","w"); foreach($urlstab $url) {     fwrite($parallel,$url+"\n"); } close $parallel;