[05:03:30] !log tools upgraded libssl on all nodes [05:03:33] Logged the message, Master [05:03:44] !log deployment-prep upgraded libssl on all salt accessible nodes [05:03:46] Logged the message, Master [05:03:56] <^d> !log gerrit upgraded libssl on phabricator [05:03:57] Logged the message, Master [05:06:35] !log tools restart nginx on tools-proxy-test [05:06:37] Logged the message, Master [05:07:03] !log project-proxy upgraded libssl on all nodes [05:07:24] Logged the message, Master [05:32:06] could someone have a look at this user-account https://wikitech.wikimedia.org/wiki/User_talk:%28euer_Kraftf%C3%BChrer%29 [05:33:42] !log project-proxy restart nginx on affected nodes [05:33:43] Logged the message, Master [05:34:01] !log deployment-prep upgraded libssl on all nodes, restarted affected ssl servers [05:34:03] Logged the message, Master [07:58:50] hashar: Did you get around to building a new labs ganglia box? Do you need any assistance with that? [15:14:06] Hey all -- brief outage of wikitech for a quick server reboot. ETC ~2min [15:19:04] And, it's done. [15:19:46] hashar: hi! [15:19:51] hashar: sorry, bad network before. [15:20:03] yuvipanda: :-] [15:20:59] yuvipanda: got time to add the jenkins jobs ? [15:21:51] hashar: sure! [15:22:16] looking at that page now [15:22:37] yuvipanda: you will want to setup Jenkins Job Builder https://www.mediawiki.org/wiki/Continuous_integration/Jenkins_job_builder [15:22:39] oh wow I need to have jjb locally [15:22:40] yeah doing [15:22:42] that is python script [15:23:07] pip install jenkins-job-builder ! :D [15:23:32] sudo apt-get remove pip python [15:23:36] er. wrong window [15:23:51] Coren, how long does the grid allow a script to run on the webgrid, before it gets killed? [15:23:57] /ignore petan [15:24:03] err leading space :D [15:24:32] hashar: execute that command and it will make your life easier (the apt-get one) [15:24:39] python is evil [15:24:57] look [15:24:58] !python [15:24:59] http://1.bp.blogspot.com/_i9u1_OaCZM4/Sgmoh8J6K-I/AAAAAAAAD30/YV6FJkJ_v6A/s400/luanda2.jpg << python waiting to eat all your resources and you [15:25:13] !del python [15:25:13] If you want to remove a key, type !python del [15:25:18] !python del [15:25:18] Successfully removed python [15:25:23] !python is is awesome! [15:25:24] Key was added [15:25:28] !python [15:25:28] aaargh [15:25:28] is awesome! [15:25:39] hmm not quite right [15:25:41] GOD NO [15:25:44] !python del [15:25:45] Successfully removed python [15:25:53] !hashar is Such a hashar! hash() [15:25:53] This key already exist - remove it, if you want to change it [15:25:56] :O [15:25:58] !hashar [15:25:58] [10:15:12] !log WMFLabs seems to have recovered now [15:26:07] !python is http://1.bp.blogspot.com/_i9u1_OaCZM4/Sgmoh8J6K-I/AAAAAAAAD30/YV6FJkJ_v6A/s400/luanda2.jpg << python waiting to eat all your resources and you [15:26:08] Key was added [15:26:28] hashar: doing now [15:26:39] oh god, petan and Cyberpower678 must be at it again [15:26:43] !python del [15:26:44] You are not authorized to perform this, sorry [15:26:47] heh [15:26:48] fine [15:26:52] yuvipanda: then you will want to clone ssh://gerrit.wikimedia.org:29418/integration/jenkins-job-builder-config.git . That is a bunch of YAML files defining our Jenkins jobs [15:26:55] * yuvipanda considers ignoring wm-bot too [15:27:06] yeah, cloning that, hashar [15:27:07] yuvipanda, no I'm not [15:27:13] yuvipanda: no sexy cloaks for wm-bot [15:27:26] or vhosts [15:27:28] whatever [15:28:10] hashar: I'm following https://www.mediawiki.org/wiki/Continuous_integration/Jenkins_job_builder [15:28:38] yuvipanda: yeah that should work :-] [15:34:33] hashar: heading for dinner, will brb in a bit [15:36:24] yuvipanda: I will be out so :D [15:37:01] can someone give this account shell access? [15:37:02] Maintenance-bot [15:45:04] Amir1: Why does a bot need shell access? [15:45:31] Usually, shell access is restricted to human beings (who can read and comply with the terms of service :-)). [15:46:08] scfc_de: because this a bot to do housekeeping of codes so It has to have access to gerrit. like L10n-bot [15:46:44] As operator of it and I promise it'll be a good boy [15:47:00] this bot is for fixing codes not running on Wikipedia [15:47:13] a bot for bots like inception [15:48:20] hashar: :) back. but I don't have to get it done *today*, so 'tis k [15:48:40] Actually I'm not sure the shell access is necessary for making patches or not [15:48:47] in gerrit [15:49:02] Amir1: Can the bot access Gerrit at the moment? [15:49:21] Amir1: no [15:49:29] Amir1: you don't need shell for gerrit [15:49:34] yuvipanda: basically add a project in mobile.yaml , then add a job: entry with something like '{name}-jslint' [15:49:41] hashar: yeah, doing that now [15:49:42] yuvipanda: poke me tomorrow if you want :] we can pair [15:49:44] scfc_de: I didn't check yet [15:49:51] hashar: almost done :) [15:50:30] petan: so you say a developer account in here is enough [15:50:33] okay :) [15:50:52] I'll bug if it was needed [15:54:34] hashar: https://gerrit.wikimedia.org/r/#/c/124617/ [15:55:02] yuvipanda: Jenkins will run some tests, one of them does a diff of the generated XML :D [15:55:27] https://integration.wikimedia.org/ci/job/integration-jjb-config-diff/84/console :D [15:56:05] hashar: nice! [15:56:15] it says 'venv: commands succeede' so I guess that is good? [15:56:44] hashar: so now I guess this needs to be merged and deployed? [15:56:51] * yuvipanda doesn't have rights for either [15:57:05] yuvipanda: https://gerrit.wikimedia.org/r/#/c/124617/1/mobile.yaml,unified :-D [15:57:09] yuvipanda: you want to use - project: [15:57:27] it will have a variable 'name' which contains 'apps-android-wikipedia' [15:57:40] then the project apply those variables to all job templates defined under jobs: [15:57:57] hashar: gah, I thought I had typed project but must've typed job because of the definition under it [15:58:02] hashar: updated [15:59:02] yuvipanda: yeah that is better :-] [15:59:12] hashar: :) [15:59:34] so that creates a job named apps-android-wikipedia-jslint [16:00:36] yuvipanda: you can deploy the job yourself since you have a LDAP account in the wmf group [16:00:45] hashar: oh. doesn't it need to be merged? [16:00:52] na :-] [16:01:03] since you have the source, you can execute JJB locally and have it create the job [16:01:03] hashar: can you please look at this comment?: https://bugzilla.wikimedia.org/show_bug.cgi?id=54710#c4 (btw ganglia really needs a stronger instance) [16:01:28] yuvipanda: command would be something like: jenkins-jobs --conf jenkins_jobs.ini update config/ apps-android-wikipedia [16:01:39] hashar: ooo nice! [16:01:43] yuvipanda: the ini file contains your Jenkins API. The wiki page explains how to get it [16:01:45] hashar: doesn't zuul-config need to be merged either? [16:01:50] hashar: yeah, I have that all set up [16:01:57] se4598: yeah I am aware ganglia needs a bigger instance :D [16:02:13] se4598: been sick today and couldn't get to it :/ ideally we would want to resize the nova instance [16:06:18] hashar: deployed! [16:07:16] yuvipanda: Zuul conf needs to be merged indeed. Then someone with appropriate permission to deploy it [16:07:36] hashar: right, so that's https://gerrit.wikimedia.org/r/#/c/124539/ [16:08:26] yuvipanda: deploying Zuul part [16:08:33] hashar: wooo! [16:09:52] hashar: are the redirects somewhere in puppet/whatever repo?, atleast WMF's ganglia stays on https, can't test for icinga (staff only httpauth) [16:12:08] se4598: the ganglia instance in labs use the operations/puppet manifests/. There must be some apache conf there [16:12:28] hashar: let me know when deploy is done. I guess I should be able to message 'recheck' to get it to run [16:12:38] yuvipanda: change deployed [16:12:47] yuvipanda: I am not sure 'recheck' will work [16:12:52] hashar: new PS? [16:12:57] yuvipanda: just amend the commit message yeah :] [16:13:22] se4598: ./templates/apache/sites/ganglia.wikimedia.org.erb [16:13:42] se4598: the name of the file is misleading, the server name is a parameter [16:14:32] hashar: workkss! :) ty [16:14:49] yuvipanda: now you can create moaaar jenkins jobs :-] [16:14:55] hashar: actually, '- https://integration.wikimedia.org/ci/job/apps-android-wikipedia-jslint/None/console : LOST ' [16:15:01] ah [16:15:14] yuvipanda: yeah sometime the jobs are not properly registered for some reason. I have found out why yet :-/ [16:15:18] :( [16:15:30] yuvipanda: fix is to go to the page, then hit [Configure] on the left and save it [16:15:35] https://integration.wikimedia.org/ci/job/apps-android-wikipedia-jslint/configure [16:15:37] doing so [16:15:44] that re register the job [16:17:02] hashar: ah, ok! [16:17:12] bug! [16:27:58] hashar: - https://integration.wikimedia.org/ci/job/apps-android-wikipedia-jslint/1/console : FAILURE in 1s [16:27:59] woo! [16:28:31] :-] [16:28:34] I am offfff! [16:28:45] hashar: thank you! [17:53:45] !log deployment-prep rebuilding simplewiki's search index optimized for the new highlighter to check the size difference [17:53:48] Logged the message, Master [18:24:12] Oh is tools having issues? [18:26:01] I cannot connect to tools-login [18:35:59] Betacommand: What's the error message? [18:53:59] I can't reach bits.beta. ping: unknown host bits.beta.wmflabs.org on 2 indepentend pc's. Does something changed on the dns config again? [18:55:04] scfc_de: host does not exist [18:59:16] Betacommand: "ssh tools-login.wmflabs.org" says "host does not exist"? [18:59:42] Betacommand: What does "host tools-login.wmflabs.org" give for you? [19:02:58] bits.beta is getting reachable again for me: bits.beta.wmflabs.org (208.80.155.137) [19:04:28] scfc_de: Im using putty [19:05:30] scfc_de: and now its working [19:05:36] Some Windows users around? How to check if DNS working there? [19:06:01] scfc_de: nslookup <...>, or 'ping

' [19:06:43] scfc_de: what do you need? [19:09:25] Betacommand: Well, if it is working now, nothing?! :-) [19:09:44] valhallasw: (Hopefully) remembered, thanks. [19:09:53] scfc_de: maybe related dns issues to what i had with bits.beta? though tools-login worked meanwhile. And now it's working, so... [19:14:44] se4598: Hard to say, but looks certainly odd; IIRC "unknown host" would imply an authorative answer from the WMF DNS server (as different from "couldn't resolve", which means the DNS server wasn't reachable). Coren rebooted wikitech about 15:20Z, and I think the LDAP server that provides the DNS records is located there as well, so that could explain some of that, but I wouldn't assume that the negative answers are cached for nearly [19:14:44] four hours. [19:21:13] I'll file a bug later to test how the DNS setup reacts (= if it reacts correctly) to wikitech being down. [19:21:24] hi, another problem, corntab for my service group [19:21:28] is deleted [19:21:44] the service group is "pywikibot" [19:22:01] Amir1: er, no, it's not. [19:22:19] Amir1: where are you looking? [19:22:44] hi valhallasw! become pywikibot and crontab -e [19:22:52] Amir1: *where* [19:23:12] tools-login in eqiad or ptmpa? [19:23:20] eqiad [19:23:24] I'm not sure [19:23:40] pmtpa is still alive? [19:23:42] because crontab -l works for me on tools-login.wmflabs.org [19:23:43] pmtpa shouldn't be accessible at all. [19:25:49] oh I get it [19:25:53] valhallasw: thank you [19:34:03] is tools down? [19:36:58] Withoutaname: not for me. What's your error message? [19:37:20] server not found error [19:37:30] firefox, if that helps [19:38:41] scfc_de: ^ another dns error it seems [19:41:36] ok just tried clearing cache [19:41:40] doesnt seem to work [19:46:18] se4598: Noted; but nothing I can do about that. The names resolve for me, and for example on Toolserver. [19:46:31] Withoutaname: for sure your pc, router and/or ISP are caching the (faulty) dns results. Will be going away sooner or later (after hours....) [19:48:11] scfc_de: ok just for filing a bug or so. Does firefox has it own dns cache? [19:50:22] scfc_de: icinga.wmflabs.org resolves (win7) on nslookup, but not on ping. On remote unix also icinga.wmflabs.org has address 208.80.155.156 but ping: unknown host icinga.wmflabs.org [19:50:22] can I fix that? [19:50:29] i just tried ipconfig /flushdns [19:50:51] And icinga domain worked 2-4 hours ago! [19:51:26] se4598: Not a clue; and very odd that ping reacts differently than other tools. [19:51:55] from cmd but I don't know if firefox has something similar [19:52:24] scfc_de: hmm, I'm having DNS issues too [19:52:31] now ping works again on remote... wtf? there's somewhere something fucked up [19:52:55] dig gives me appropriate response but mosh doesn't [19:53:06] yuvipanda: try again? I had intermittent mosh failures [19:53:17] valhallasw: hmm, works now [19:53:53] cleared cache and did flushdns [19:53:56] anything else? [19:59:07] What does it mean when "host <.....>" doesn't give an output but simply returns? Happens to me on one maschine for "host icinga.wmflabs.org" [20:00:22] Does that mean that a dns server gave a (positive) answer but not an dns entry? [20:01:40] eh, did someone accidentally simplewiki? [20:08:06] what simple? [20:08:51] the late http://simple.wikipedia.beta.wmflabs.org [20:10:22] there seems to be some spambots... but what was your question? [20:10:53] well, it was there, now there isn't:P [20:11:38] Special:RandomPage works. http://simple.wikipedia.beta.wmflabs.org/wiki/Colloquy_%28IRC_client%29 [20:12:01] "It isn't there" isn't very precise [20:13:28] MaxSem, there are some DNS problems for some of us, saying wmflabs domains can't be resolved, if you refering to that [20:13:35] ah [20:19:08] anomie, is it normal for a bot, on a single run, to attempt to edit 3,221 articles, and edit conflict on every single one of them? [20:19:36] anomie, have any changes been made to the API recently? [20:20:27] Cyberpower678: That sounds odd. Nothing much changed in the API recently though. [20:20:38] Cyberpower678: are you running multiple bots in parallel? [20:20:48] I've seen issues with edit tokens in that case [20:21:25] Cyberpower678: For enwiki, not since Thursday. Today's deploy to non-Wikipedias seems to have not happened (yet) either. [20:21:59] anomie, sorry anomie I got kicked from Freenode for some reason. All I got was Cyberpower678: That sounds odd. Nothing much changed in the API recently though. [20:22:25] Cyberpower678: For enwiki, not since Thursday. Today's deploy to non-Wikipedias seems to have not happened (yet) either, so nothing there since Tuesday. [20:23:16] Cyberpower678: I'm not completely sure how edit conflict detection works, but I've had issues with edit tokens when running two bots in parallel under the same name [20:23:36] How very odd. Nothing has changed with Peachy, or my bots in regards to editing pages. [20:24:00] But recently these strange things have been happening. Editconflicts and badtokens. anomie [20:26:39] valhallasw, I've been having issues with edit tokens. It seems the token generator is a bit faulty in regards to that. However, this was never an issue before. [20:27:12] I've been running SoxBot scripts on Cyberbot I since March 2012 the same way I do now and they never had any issues. [20:29:55] Cyberpower678: the issue is that you only get a single edit token -- so if two bots request an edit token, they both get token 'A' [20:30:10] but you can only edit once [20:30:30] valhallasw, wtf? That was never an issue before. [20:43:46] * Cyberpower678 needs to deploy the Peachy to force refresh tokens on edit failure. [21:09:08] Hey all. Is there a master listing of all namespaces across all WMF projects, and what their number is (i.e. 0 for mainspace, 1 for mainspace talk, 6 for file, 10 for template, etc.) ? I was told there might be one here. [21:10:51] Sven_Manguard: there is a master api for that :P [21:11:12] petan: I am not sure what you just said. [21:11:41] if it exists in readable list form, I would like to see it though [21:11:55] Sven_Manguard: http://www.mediawiki.org/w/api.php?action=query&meta=siteinfo&siprop=namespaces [21:12:26] every wiki has own namespaces this api will get them [21:13:46] there are 76 namespace numbers in use across all projects. That has 34. [21:14:02] yes as I said every wiki may have different namespaces [21:14:14] you need to run it on wiki where you want to know it [21:14:19] it can also change and it does change [21:14:39] yeah, but if all I have is the number, finding which project that number is used on will be impossible... [21:14:45] it is also possible that namespace XX has id 20 on wiki 1 and id 26 on wiki 2 [21:15:26] you could run this api on all wikimedia project and summarize these, but you will likely find conflicts [21:15:44] eg. different NS with same ID [21:29:37] Sven_Manguard: On the Toolserver there is a table that has all this data, and https://bugzilla.wikimedia.org/48625 is about providing that in Labs as well. If you just need a dump of that, I can upload that somewhere. [21:30:08] scfc_de: that would be awesome, yes please [21:33:09] Sven_Manguard: http://toolserver.org/~timl/toolserver.csv is a tab-separated, first row = column names dump of the Toolserver data as of now. [21:35:57] thanks [21:37:04] scfc_de: can you do that where the number is the first field, so that I can drag the whole thing into excel and let me sort by number? [21:39:52] Sven_Manguard: just ctrl-a ctrl-c ctrl-v it into excel? [21:39:59] then sort on column ns_id [21:40:35] (make sure to set your browser to utf-8, though, unless you like seeing ÐÑˆÐ°Ð±Ð»Ð¾Ð½ instead of Ашаблон) [21:40:38] valhallasw: because it treats everything as being in one column [21:41:48] Sven_Manguard: oh, that's silly. it does work if you first paste it in notepad, then ctrl-a ctrl-c it again, then ctrl-v in excel [21:42:04] Sven_Manguard: I can also mail you the xlsx [21:42:25] I PMed you my email [21:42:53] Yep. Should be in your inbox in a few seconds. [21:44:24] got it [21:54:56] se4598: actually I dont know if this is relevant but downforeveryoneorjustme.com is also reporting errors, I'm guessing this problem is location specific [21:57:23] scfc_de: do you opened a bug report for that dns thing? [21:57:50] What is the CNBanner namespace? [21:59:21] Withoutaname: just pinged ganglia (a not already tested domain) first try: unknown host, 3 seconds later second try: it works. I don't think thats location specific and that result let me guess that the problem is still exists somewhere [22:00:03] Sven_Manguard: guessing that's Meta's namespace for [[meta:Special:CentralNotice]] [22:00:10] se4598: Not yet. [22:01:03] Withoutaname: A ton of projects have it. What you say makes sense though. [22:01:49] a lot of projects also have the "steward" userright and other global rights that don't make sense [22:02:09] considering "steward" as a right only appears in CentralAuth and on Meta [22:04:43] maybe all the local projects have their own Special:CentralNotice but no one can use it because its controlled from Meta only [22:06:23] Sven_Manguard, it is the namespace Centralnotice uses for banner translation -- I thought I fixed a bug a long time ago where we only register it on meta (or master wikis); but I could be wrong [22:08:14] hmm [22:08:48] I am seeing "Course" and "Institution" namespaces on EnWiki, but it appears that... what? [22:09:21] they can't be edited? [22:09:25] do they actually exist? [22:10:01] I think they're special namespaces [22:10:08] try them out on en.wikipedia.beta.wmflabs.org [22:14:26] scfc_de: https://bugzilla.wikimedia.org/show_bug.cgi?id=63709 [22:15:39] Withoutaname: ^ filed [22:16:10] Withoutaname: they exist only on EnWiki and TestWiki, but I don't see what they do [22:16:36] se4598: Thanks! [22:16:39] and since they don't appear editable, I don't need to worry about them [22:19:05] Crap, my power is flickering. If I drop out before I'm ready to leave, thanks to everyone that's helped me. [22:19:30] If not, well I was going to thank you later but I guess I just did so now... [22:19:45] where are thye [22:20:00] i cant see them at https://en.wikipedia.org/wiki/Course:foo [22:21:16] i always thought course and institution ns were part of the education program [22:21:32] Withoutaname: they're in the list of namespaces, but I don't see pages [22:22:22] well yes but [22:22:57] meta also has a ton of namespaces no one else has [22:23:03] if they actually appear as namespace pages on enwiki, [[Course:foo]] would automatically redirect to [[Course:Foo]] [22:23:09] 36 overall [22:23:21] mostly translation [22:23:24] lemme check [22:23:24] 36 editable* namespaces overall [22:23:29] Sven_Manguard: Just to clarify: I am not sure how the Toolserver data was generated, i. e. if every x days they start with nothing and iterate over all APIs, or if the list has been maintained manually, or if at some point in time the MediaWiki configuration for enwiki had those entries and for backwards compatibility they were not deleted in the table afterwards, etc. So for authorative answers, you should query the API. [22:24:30] scfc_de: I already have a list of every namespace number that the edit counter supports, I just needed to know what projects were using those numbers [22:24:35] I see some namespaces point to "Grants" and "Research" [22:24:38] so I'm not worried about gaps [22:24:41] metawiki_p meta.wikimedia.org 200 Grants [22:24:42] metawiki_p meta.wikimedia.org 201 Grants talk [22:24:44] metawiki_p meta.wikimedia.org 202 Research [22:24:45] metawiki_p meta.wikimedia.org 203 Research talk [22:24:47] metawiki_p meta.wikimedia.org 204 Participation [22:24:48] metawiki_p meta.wikimedia.org 205 Participation talk [22:24:49] I think those committees had separate wikis folded back into meta [22:24:50] metawiki_p meta.wikimedia.org 206 Iberocoop [22:24:51] metawiki_p meta.wikimedia.org 207 Iberocoop talk [22:24:53] metawiki_p meta.wikimedia.org 208 Programs [22:24:54] metawiki_p meta.wikimedia.org 209 Programs talk [22:24:55] legoktm: https://tools.wmflabs.org/globalprefs/ <- no webservice? [22:24:56] oops, sorry [22:25:05] Sven_Manguard: why are you talking about Meta-Wiki namespaces? [22:25:39] huh: I am changing the colors on the edit counter because there are problems [22:26:08] for example, that many of the new ones are beige [22:26:15] Which one? [22:26:20] Which editcounter? [22:26:31] cyberpower's [22:26:35] the rebuild of X!s [22:26:43] X!'s [22:28:43] yeah those namespaces were previously wikis belonging to separate committees [22:28:58] like strategy.wikimedia.org they were folded back into meta [22:29:00] my guess [22:30:44] Sven_Manguard: btw where did you get the namespaces from? API? [22:31:04] [14:33] scfc_de Sven_Manguard: http://toolserver.org/~timl/toolserver.csv is a tab-separated, first row = column names dump of the Toolserver data as of now. [22:31:10] that [22:31:36] oh [22:32:17] huh, I thought you were here when you saw that [22:32:35] ? [22:32:49] i didn't see you enter chat [22:32:57] nevermind [23:33:08] huh are you still here [23:33:17] Yes. [23:33:56] Withoutaname: where else would I be? [23:34:26] can you move https://test.wikipedia.org/wiki/File:Aa.svg.png [23:34:47] to say https://test.wikipedia.org/wiki/File:Aa.svg.png [23:35:01] i dont have filemove rights [23:35:17] Withoutaname: File:Aa.svg.png with or without the hidden char? [23:35:32] without the hidden char [23:35:35] ok [23:35:41] how did you know there was a hidden char [23:36:12] It displayed weirdly [23:36:17] BTW, can't move it [23:36:27] You do not have permission to move this page, for the following reasons: [23:36:29] Source and destination titles are the same; cannot move a page over itself. [23:36:30] A page of that name already exists, or the name you have chosen is not valid. Please choose another name. [23:37:15] so mw doesnt detect it in filenames [23:37:21] Withoutaname: hey, there is no hidden char! [23:37:24] You lied to me [23:37:38] https://test.wikipedia.org/wiki/File:Aa.svg.png (just normal text) [23:37:57] yes I put in a lot of hidden char between the first a and the second [23:37:58] BTW is that a copyvio? [23:38:14] idk i marked it for deletion just in case [23:38:18] Withoutaname: there is none there currently, check [23:39:26] huh: example: https://en.wikivoyage.org/w/index.php?diff=2544908 [23:39:38] apparently mw doesn't even see it in diffs [23:43:26] or maybe mw stripped them when I uploaded it [23:57:52] Withoutaname: Where in the diff would you expect a difference? [23:58:46] I mean at least the diff shows up otherwise there would be no green/yellow highlights [23:58:59] in there use view-source [23:59:05] and find the text changed [23:59:17] the diff marker will show the character removed