[14:12:18] harej, re. high performance API, it certainly depends on what is happening behind the API :) [14:13:03] halfak: I am going to create an API that will convert one identifier into other identifiers, irrespective of whether it's on Wikidata. This should help with consolidating your report. The API will be optimized around exactly one thing: taking one identifier and giving you the other identifiers. [14:13:46] It would be nice if I could use such a service in order to convert a lot of IDs in batch. [14:13:53] How many DOIs are there? [14:14:04] How many DOIs would one major journal have? [14:15:14] It will let you do that, yes. [14:19:12] Essentially, all the conversions will be done in advance; the API serves them to you. [14:59:01] harej, but numbers are key [14:59:23] If I need to process 10M IDs, I'd like to be able to do that in a day or two without taking the service down. [15:00:26] I don't think you can send 10 million at once. But with POST requests you could send a lot? [15:01:26] ha. Not at once, but through some process. E.g. one request per DOI or maybe 50 DOIs per request. [15:01:48] With ORES, you can score about 2M edits per day without affecting overall performance substantially. [15:02:04] With two parallel requests for 50 revs each. [15:03:37] One DOI per request would slow you down a lot. I will want to be able to shove as many things into the query as possible to cut down on HTTP overhead [15:04:12] For instance, does Crossref let you query more than one DOI at a time? [15:36:12] harej, not sure on that one. [15:36:37] Some reasonable limit should be useful. For ORES, I like 50. We get about a 3x speedup per item when batching at 50 [15:36:47] The speedup is not substantially different at 100. [15:36:51] So 50 is what we chose. [16:11:32] halfak: I will let you define your own limit, so long as it's not higher than the limit I set, obvious. [16:11:50] But bulk querying was an important consideration.. [17:18:46] also, halfak, what do you know about system tuning for improving web server performance? Not just at the web server level but at the system level [17:56:36] halfak: it looks like it's only you and me for the backlog grooming [17:56:57] schana, should we skip it this week? [17:57:10] fine by me, unless you had something [17:57:14] harej, depends on what the web server is doing. [17:57:17] schana, nope [17:57:18] :) [21:01:27] o/ AbbeyRipstra