Switch to auth api usage for github for 100X traffic support

[grssam]

Right now, the limit is 60 calls per hour. If we switch to auth api calls, we can have 5000/hr. I think Sankha has an idea on how to do this..

[ngsankha]

Yes, I do. But that would require having API keys, which is unsafe if we keep them openly on the client side. But if we don't want to do that, then its another headache to let a user login to get access to his API keys to make the calls via his ID. Without a proper server infrastructure, I suggest we close this!

[debloper]

1. The frontend wouldn't make the call anyway, going forward. There will be a service running to get that data, serialize it & create consumable JSON - we'll be having backend for that, which can safely have the API Tokens & Secrets.
2. Now that we're at it... having the cron running to give quasi-realtime JSON, why'd we need more than 60 API calls per hour? That ought to suffice, IMO.

[grssam]

Because even one run of the cron job can lead to more than 60 calls in one go. Assume 10 tracked GH repos, 100 users, it is highly likely that more than 60 calls will be made.

[debloper]

With https://api.github.com/users/:user/events in hand, it beats me why are we going for repo-specific search - where the list/count of repositories will always be incomprehensive.

From the user's public events, we filter out only the ones performed on /github\.com\/mozilla(.*)/ and we get what we need with one single call.

Possible, we might or might not require ALL that we need. But for the time being, this appears to be good enough. Plus, always remember - these are to be done with a daemon & not by Frontend with AJAX calls. So, consider the advantages/pitfalls according to that.

[ngsankha]

We looked into it. But it doesn't show the entire history, so lets say I have a 6 month old commit, it will not turn up using the API call.

[grssam]

@debloper The solution you are proposing requires minimum of #users calls (which itself can be more than 60) on top of that, the API is paginated, so we will not get everything in one page, while the solution we are following requires maximum of #users * # repos calls but a minimum of only #repos calls. Even then, as said by @sankha93 above, the users API does not give every commit.