[17:27:34] Greetings, Wikimedians. I am developing a tool that will utilize the Wikimedia Oath framework. While it is technically feasible to retrieve a user's email address via Oath Scope, are there any restrictions or concerns regarding the collection or storage of email addresses within tools under Toolforge's privacy policy? Alternatively, is it sufficient to rely [17:27:34] solely on the username? [17:39:47] for sure [17:42:59] anafibnshahibul: for a sensitive topic like this; suggesting to ask on the mailing list instead [17:47:56] Hello, I have a list of around 50 wikidata items i want to download in json format, what is the quickest and easiest way to do so? [17:48:18] 0 [17:49:05] you can look at how we do it at the top of this file : https://github.com/slashme/parliamentdiagram/blob/master/src/static/js/arch.js [17:49:30] Wikidata didn't like this script, it told me something about user agent but don't know exactly how I would need to change it to make it work: [17:49:30] ``` [17:49:32] #!/bin/bash [17:49:33] readarray -d '\n' -t is < <(perl -gne 'print "$&\n" while /Q[0-9]*/g' da.txt) [17:49:35] for i in ${is[@]}; do [17:49:36] curl 'https://www.wikidata.org/entity/'"$i".json -L -J -O [17:49:38] sleep 1 [17:49:39] done``` [17:50:53] All requests to Wikimedia servers need to be accompanied by a valid user agent that includes the name of the program you're running, the version number, and an email address (re @Alessandro: Wikidata didn't like this script, it told me something about user agent but don't know exactly how I would need to change it to ...) [17:51:07] substitute "curl ..." with 'curl -H "User-Agent: Alessandro Griseta" ...' or something (re @Alessandro: Wikidata didn't like this script, it told me something about user agent but don't know exactly how I would need to change it to ...) [17:51:41] is there a toolforge app that allows me to do this? I think it would be better [17:51:55] is there a git repo you always wanted to be included in codesearch but hasn't been yet? let me know. I kind of need one to test :) [17:52:24] is there something you're doing in your script besides simply getting the content of an item? (re @Alessandro: is there a toolforge app that allows me to do this? I think it would be better) [17:52:49] no literally just item in .json format [17:52:51] maybe you should use an existing wiki bot [17:53:33] (I'm pretty sure most Toolforge tools also expect a proper user agent) (re @mahir256: is there something you're doing in your script besides simply getting the content of an item?) [17:54:17] you can use https://www.wikidata.org/wiki/Special:ApiHelp/wbgetentities and get multiple items at once (re @Alessandro: no literally just item in .json format) [17:55:00] @Alessandro, this advice might be coming a bit late but you're just doing this as a one-off, you might have good luck with PAWS and pywikibot. https://hub-paws.wmcloud.org/hub/login [17:55:31] In that case the queries will be made using your wiki SUL account. [17:56:31] PAWS is not as flexible as toolforge but it saves the trouble of venvs, container builds, auth, etc. [17:56:37] in many cases at least [17:57:21] Ah I see, I thought there was a toolforge app just to download as json or something, I will see what I can do on PAWS (re @wmtelegram_bot: PAWS is not as flexible as toolforge but it saves the trouble of venvs, container builds, auth, etc.) [17:57:56] oh, there might be and existing tool. But if you're thinking about making your own tool then... paws will be a lot easier. [17:57:58] if you know what downloading a single item in json would look like in code I can extend to my 50 items :) [17:59:03] query.wikidata.org offers JSON downloads (re @Alessandro: Hello, I have a list of around 50 wikidata items i want to download in json format, what is the quickest and easiest way to do s...) [17:59:24] take the first example listed in the ApiHelp link I gave, and replace "Q42" with a pipe-separated list of items (e.g. "Q1|Q2|Q3|...") (re @Alessandro: if you know what downloading a single item in json would look like in code I can extend to my 50 items :)) [18:00:00] the resulting URL, combined with a proper user agent as noted by James and I, should allow your bash script to work (re @mahir256: take the first example listed in the ApiHelp link I gave, and replace "Q42" with a pipe-separated list of items (e.g. "Q1|Q2|Q3|...) [18:00:52] that's more helpful than anything I have to offer :) [18:01:52] cool, this should work :) [18:05:15] in the end all that was necessary was `-A 'email address, name etc.'`, although it was great to hear from all of you, it's hard to stop learning in Wikimedia world lol [21:46:49] dancy: if you have a few minutes, can you retest T429978 with Trixie? I built a new base image which I suspect will fix both the bugs you reported. [21:46:49] T429978: Project members cannot ssh into newly created deployment-prep instances - https://phabricator.wikimedia.org/T429978 [21:47:09] (I haven't updated Bookworm yet, that's in process) [21:48:47] andrewbogott: Sure thing. I'm about to step out for a break but I'll get a new instance going before that. [21:48:56] ty! [21:49:21] time-to-login should be drastically shorter now as well, maybe 3-4 minutes [21:50:03] new upstream packages make it slower and slower until we refresh [21:50:25] ok I fired up deployment-dancy2. I'll check on it later [21:50:59] sounds good!