Instagram Helper (Scraper) Chrome Extension
Now the extension allows to limit the amount of fetched users to Nth first returned users. From my observation, the newest followers/following users are returned in the beginning, therefore limiting the output could be useful to get the newest relationships only, especially for accounts with huge amount of relationships. If limit value is 0 and it is the default value on opening the extension popup window, the limit is not applied. The limit is applied independently to both relationships: followers and following, it means if limit is 1000, the extension fetches 1000 followers and 1000 following persons when requested relationship is All (followers and following). Therefore the length of created list could be about 2000 users, depending on how much followers and following intersect. Also the setting limit to 1000, doesn't mean that exactly 1000 users will be returned in the output, typically one response from Instagram.com contains about 18-20 users, therefore the total amount of returned users could be a little bit higher than asked, but not more than 20.
Also to return the results faster, the results are returned just when all list is fetched for Instagram.com. However these result are not detailed enough, they just contain the username and small picture of the profile. Therefore getting the detailed info continues in background, and when it completed, the list with detailed info is displayed.
I've just uploaded the new version to Chrome Web Store. It is aligned to the new API from Instagram. However, as far as I know, the new API returns the detailed information about user, e.g. amount of posts, followers, and following persons, only in response for single user request, therefore for each found user the individual request should be sent to collect the detailed information. It slows down the process.
I've just discovered that Instagram has changed the API and endpoints. Therefore the main functionality of the extension doesn't work now. I am motivated to adjust code as soon as possible, hopefully will be able to publish the updated version this weekend.
Version 0.290 is uploaded to Chrome Web Store. It allows to export the results into Excel file. Also it contains more improved handling of HTTP 429 error - now user interaction is not required for resuming the process, and status bar gives more clear information what happened and when retry will be done. In addition, some minor bugfixing and improvements are done, including indicating the case when actual amount of returned users is different than amount displayed on profile info; I noticed it could happen with users having more than several thousands connected users, maybe the amount of followers and following users on profile info is updated asynchronously or with some delay. This is a stable version, now I am not planning to develop the new versions in the next several weeks, as I got something else to do; however sooner or later I will return to this extension, as still there are some planned improvements.
Version 0.280 is uploaded to Chrome Web Store. The main changes:
Yesterday I uploaded an new version of my extension into Chrome Web Store. The main changes are:
The main changes are:
I am going to publish a new version of the extension this weekend. The main changes are:
I've just published a new version 0.23 of extension on Chrome Web Store. So when you click the extension icon, the displayed popup window has two buttons:
1) Get Insta Users (manual fetching). The previous approach of getting the list that requires the manual fetching, this option will be removed soon.
2) Get Insta Users. The new approach that creates a list for the username specified in the input control.
There are two options for autofetching
1) "Page size for fetching users" - amount of users to be returned in one click. The default value is 100, however debugging this functionality, I realised that instagram server doesn't respect this parameter too much, and usually returns about 20 users in one call.
2) "Interval between requests for fetching users, ms" - delay before issuing the next request when the processing of previous request is completed. If there is no delays, for fetching the long list of users, the instagram will start to return HTTP code 429 (too many requests, try several minutes later). Right now this situation is not handled, and the only option to avoid that is having a delay.
I am eager to finish a new version which allows to get rid of manual fetching of following users/followers. However because of private businesses, it is not going to happen this weekend, and now is planned for the next weekend (March 11-12). In the meantime, I was able to send AJAX requests that fetches the followers list. It is a sad, but Instagram checks the Referer HTTP header, and doesn't return the data if Referer is not instagram - that is the case when I send the requests from the page generated by extension. I was tried to inject the code that sends requests into Instagram HTML page and it worked, however looked quite ugly and cumbersome. Therefore finally I decided to go with the option when this HTTP header is modified before sending the request using Chrome API, however it required to grant the additional permissions for the extension, and switch from "event page" to "background page" approach.