Thursday, 2016-09-29

openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from global requirements  https://review.openstack.org/37890000:11
*** armax has joined #openstack-requirements01:46
openstackgerritRussell Bryant proposed openstack/requirements: Update ovs to 2.6.0.  https://review.openstack.org/37906701:51
*** armax has quit IRC02:31
*** david-lyle has quit IRC03:04
*** armax has joined #openstack-requirements03:22
*** armax has quit IRC03:34
openstackgerritMerged openstack/requirements: Updated from global requirements  https://review.openstack.org/37890004:34
*** coolsvap has joined #openstack-requirements04:42
openstackgerritMerged openstack/requirements: Update oslo.policy to >= 1.14.0  https://review.openstack.org/37894604:45
openstackgerritMerged openstack/requirements: Update ovs to 2.6.0.  https://review.openstack.org/37906704:45
openstackgerritTony Breeds proposed openstack/requirements: Remove non-existent lower bounds  https://review.openstack.org/37915805:55
tonybdirk: nice one getting the uc-check "working"  It now looks like we have at least one bindep issue06:35
tonybdirk: we also need to backport the environment name change to stable/* http://logs.openstack.org/16/361116/2/check/gate-requirements-tox-py27-check-uc-ubuntu-trusty/c2d6a1d/console.html#_2016-09-29_05_29_47_29107906:37
dirktonyb: yep, realized that as well06:38
tonybdirk: but we can do the rename in a single commit there as the infra stuff has landed already06:38
dirkWill do in a few min06:38
dirktonyb: yeah, i'll Look into the bindep issue06:38
tonybdirk: cool.  I can +W it please make sure you mention the 2 master ChangeIDs06:39
tonybdirk: http://logs.openstack.org/18/376218/2/check/gate-requirements-tox-py27-check-uc-ubuntu-xenial/0ac84b7/console.html#_2016-09-29_05_29_21_131170 (in case you hadn't seen it)06:40
openstackgerritTony Breeds proposed openstack/requirements: Quick wins for python3 support  https://review.openstack.org/37919306:49
openstackgerritDirk Mueller proposed openstack/requirements: DNM: Fixes for check-uc jenkins job  https://review.openstack.org/37920306:57
dirktonyb: yay!07:08
dirktonyb: meaningful errors..07:08
tonybwoot!07:08
dirkContextualVersionConflict: (Sphinx 1.3.6 (/data/dmueller/src/Cloud/requirements/.tox/py27-check-uc/lib/python2.7/site-packages), Requirement.parse('sphinx!=1.3b1,<1.3,>=1.2.1'), set(['openstack-doc-tools']))07:09
dirkContextualVersionConflict: (Sphinx 1.3.6 (/data/dmueller/src/Cloud/requirements/.tox/py27-check-uc/lib/python2.7/site-packages), Requirement.parse('sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2'), set(['os-api-ref']))07:09
dirkContextualVersionConflict: (pyparsing 2.1.9 (/data/dmueller/src/Cloud/requirements/.tox/py27-check-uc/lib/python2.7/site-packages), Requirement.parse('pyparsing<=1.9.9'), set(['pulp']))07:09
dirkso openstack-doc-tools and os-api-ref is easy, we just need new releases there07:09
tonybdirk: Awesome!07:09
dirkPuLP seems to be some external stuff07:09
tonybdirk: Yeah, so our specification for pyparsing is more liberal that pulp's interesting07:10
tonybdirk: is that all the problems or just a snippet?07:11
dirktonyb: it is all07:15
tonybdirk: Nice.07:17
dirktonyb: hmm, we should add os-api-ref to the projects.txt07:17
dirkfor openstack-doc-tools I requested a new releaes07:18
tonybdirk: We can manually sync os-api-ref07:18
dirktonyb: true.. done. when that merges, we can request a new release07:22
tonybdirk: cool07:22
dirkpulp is the most painful it seems,b ut it is only used in congress07:23
tonybYeah that's going to be a bit of a pain.07:26
tonybdirk: it looks like the next version of pulp will work as it reqires pyparsing>=2.0.107:29
tonybdirk: so we might just need to fail until then07:29
dirkyeah, would be nice to have a kind of "blacklisting-until-new-release-is-there" capability07:30
tonybdirk: Oh huh they only cap pyparsing on python207:30
tonybOh and dims fixed it07:30
dirksmall world :)07:30
tonybdirk: we could do something horrible like remove it from the venv before we run the check07:36
tonybdirk: Or les gross just skip verifying PuLP07:37
tonybdirk: soemthing like: http://paste.openstack.org/show/583418/07:39
tonybdirk: with comments etc07:39
dirktonyb: right, I was thinking more about ignoring specific requirements from specific packages07:42
dirkso that it can error out when a particular problem isn't there07:42
dirkso rather than blacklisting before, blacklsting the error message07:42
dirkwhich is just making it easier to consume and then comparing it with a known-issues-uc.txt07:43
dirkbasically this is XFAIL handling - we expect a fail, so we need to fail when the fail isn't there07:44
dirkI'll draft a patch for that when meeting madness has ended07:44
*** tonyb_noSSH has joined #openstack-requirements07:48
tonybdirk: cool07:48
tonybdirk: Sorry I went quiet my IRC connection dropped :(07:48
*** strigazi_AFK is now known as strigazi08:29
*** jpena|off is now known as jpena08:57
openstackgerritgengchc2 proposed openstack/requirements: Use ConfigParser instead of SafeConfigParser  https://review.openstack.org/37526408:57
openstackgerritChris Dent proposed openstack/requirements: Bump gabbi to 1.26.1  https://review.openstack.org/37934710:25
openstackgerritDirk Mueller proposed openstack/requirements: Fixes for check-uc jenkins job  https://review.openstack.org/37920310:45
openstackgerritChandan Kumar proposed openstack/requirements: Update os-testr to 0.8.0  https://review.openstack.org/37937011:00
*** openstackgerrit has quit IRC11:19
*** openstackgerrit has joined #openstack-requirements11:19
*** jpena is now known as jpena|lunch11:43
dimsdirk : you need this quickly? https://review.openstack.org/#/c/379216/12:25
number80dims: I assume that yes, if suse ships newer sphinx (and they do if I remember correctly)12:25
dimsnumber80: i see12:27
number80I haven't done that update in CentOS because of that issue12:27
dimsnumber80 : gotcha, talking to dhellmann on #openstack-release12:29
openstackgerritcaoyuan proposed openstack/requirements: Remove the unnecessary "# BSD"  https://review.openstack.org/37942012:29
number80dims: thank you :)12:29
openstackgerritcaoyuan proposed openstack/requirements: Remove the unnecessary "# BSD"  https://review.openstack.org/37942012:45
*** david-lyle has joined #openstack-requirements12:57
*** jpena|lunch is now known as jpena12:59
dirkdims: I don't realy need it quickly, but it would be nice to clean this up13:20
openstackgerritMerged openstack/requirements: Remove now unused py27-with-upper-constraints env  https://review.openstack.org/37873413:21
*** breton has joined #openstack-requirements13:22
bretonhi13:22
dimshey breton : what's up?13:23
bretonwe have bug 1628883 in keystone, which happened because of too low lower-constraint in requirements.txt13:23
openstackbug 1628883 in keystone (Ubuntu) "Minimum requirements too low on oslo.log for keystone" [Undecided,Triaged] https://launchpad.net/bugs/1628883 - Assigned to Corey Bryant (corey.bryant)13:23
bretonis it safe to just bump it like https://review.openstack.org/379451 or we need to do something else in openstack/requirements?13:23
dimsbreton : we won't touch stable/newton requirements as its too late13:25
dimsbreton : i'd advocate a bump in requirements master branch if that's not done yet13:27
bretondims: that's already done13:27
dimsperfect13:27
dimsbreton : if we bump g-r for stable/newton then we will have to cut releases *all* projects. so we have to avoid that13:28
dimsbreton : the guidance to packagers is to make sure they are as close to upper-constraints as possible, so that will minimize impact13:29
bretondims: cool, thanks.13:30
dimsdirk : if you patch the version #, we can roll it out https://review.openstack.org/#/c/379216/13:33
dirkdims: yep, sorry, I just read the backlog on this13:34
dirkdims: let me quickly check with aj and I'll do that in a min13:34
anteayadirk: why do you +1 an incorrect defintion on the mailing list?13:39
anteayait is hard enough to help folks as it is, encouraging them to use incorrect terms doesn't help13:40
dirkanteaya: hmm, I guess i am confused then as well13:40
anteayathat is fine if you would like some clarification13:41
anteayaI'm happy to assist13:41
anteayabut encouraging folks to use slang and incorrect terms just ensures they get little assistance in infra13:41
dirkanteaya: argh, my bad, I think I quoted the wrong thing :(13:42
anteayait is exhausting to have to spend a good chunk of every request for help educating the querant on what their actually asking about13:42
dirkanteaya: I see what you mean.. I wanted to +1 the "promote..to.voting" part13:42
anteayadirk: if you could be so kind as to take some time when you have some space and clarify it would mean a lot to me13:42
anteayajust take your time13:42
anteayaand thank you13:42
anteayaand so that you and I are clear13:43
anteayagate is a pipeline of which we have many13:43
anteayathe name of the pipeline is at the top of every queue: http://status.openstack.org/zuul/13:44
anteayacheck, gate, post and so on13:44
anteayajobs return results, success or failure13:44
anteayaverified +1, -1, +2 or -2 are recorded on a per job basis depending on the outcome of the job13:45
anteayapermissions on what can leave verified votes is set in gerrit13:45
anteayathird party ci systems can not leave verified +2 or -2 on patches13:45
anteayaas that would prevent patches from mergeing based on third party ci sytem job results and we don't allow that in our system13:46
anteayaonly infra's ci results may leave verified +2 or -1 on our patches13:46
anteayaand thanks13:46
dirkanteaya: sec, just trying to wrap up the meeting in openstack-meetings-alt, brb13:47
anteayasorry to disturb13:48
anteayaonly infra's ci results may leave verified +2 or -2 on our patches*14:05
dirkanteaya: ok, am back14:05
anteayathird party ci systems may leave verified +1 or -1 on a repo's patches, if the repo team allows them14:05
anteayadirk: welcome back14:06
anteayajust fixed a typo I made above14:06
dirkanteaya: so basically what we want is that the two 3rd party gates can do a "jenkins -1" on a PR  check14:06
dirkso I guess the proper thing would be "allow them to vote on a check gate"14:06
dirkright?14:06
dirkor is that the check-job in the gate?14:06
* dirk needs a picture :)14:06
anteayatwo third party jobs, not gates14:08
dirkthird party ci's that have third party jobs, yes14:08
anteayaI don't want anything personally, but the email is about allowing third party ci to leave verified +1 or -1 on patches, is my hunch14:08
anteayathere is no such thing as a check gate14:08
anteayathere is a check job and a gate job14:09
anteayajobs can run in check, they can run in gate, they can run in post14:09
dirkwell, above you said "gate is a pipeline of which we have many"14:09
anteayajobs run in pipelines14:09
anteayayes14:09
dirkso we have a check pipeline, and a gate pipeline14:09
anteayawe do so14:09
dirkand we talk about the check pipeline14:10
anteayawe do, we also talk about other pipelines to14:10
dirkso for me gate and pipeline is interchangeable14:10
anteayathat can work until it doesn't14:11
dirkright, ther eis a gate gate :)14:11
anteayapipeline is defined here: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n414:11
dirkthat makes it quite difficult14:11
anteayawhat makes what difficult14:11
dirke.g. "check pipeline" and "gate pipeline" is easier for me to understand than14:11
dirk"check gate" and "gate gate"14:12
anteayaI'm glad the correct way is also easiest to understand14:12
dirk:-)14:12
anteayapipelines tell our tooling which set of jobs to run14:12
anteayawe run the jobs considered check jobs in the check pipeline14:13
anteayagate jobs in the gate pipleline14:13
anteayapost jobs in the post pipeline14:13
anteayacheck and gate are closely related14:13
anteayain that all the jobs that run on a patch in the gate pipeline have to have run and passed in the check pipeline14:13
anteayaalso we want all jobs that vote in check also vote in the gate pipeline14:14
dirkaha!14:15
dirkthats an important piece of info14:15
dirkI was wondering about that as well14:15
anteayaI'm so glad we are having this conversation14:16
dirkanteaya: hmm, so basically we can not have voting 3rd party hosted checks ?14:16
anteayaare things starting to make sense?14:16
dirkyes14:16
anteayagreat14:16
anteayawell third party ci is separate from our infra discussion14:16
anteayaso it appears we share an understanding of the names of things in the infra workflow, yes?14:17
dirkyes14:17
dirkto be honest the result of our discussion should be long in the z uul reference manual14:17
dirkI was scratching my head over that one already several times14:17
dirkI am aware of the issue with not relying on 3rd party jobs in the gate/post pipeline14:19
dirkthat makes sense to me14:19
anteayagreat14:19
dirkthe part I'm lacking on is why there should be a strong relationship between what is voting as a check and voting as a gate job14:19
dirkwhy can't there be more voting checks than gate jobs ?14:20
anteayanow if a given project wishes to allow third party ci to leave a verified +1 or -1 on a patch, a project can do so14:20
anteayathis is the neutron-ci group: https://review.openstack.org/#/admin/groups/510,members14:21
dirkbasically what we want is "if any of the 3rd party ci jobs fail, the overall jenkins vote should be a -1, so that it is dead easy to see on the gerrit listing that this review is no good"14:21
anteayathis gerrit group is administered by the neutron team14:21
anteayathis group gives permissions to those ci accounts listed to leave verified +1 or -1 on neutron patches14:21
dirkbut we don't require them to be added to the gate pipeline *yet*. there is an independent effort to rebuild the 3rd party cis on the openstack hosted infra, and then they can be voting on the gate pipeline14:21
anteayawhen you say basically what we want is, who is we?14:22
anteayaI don't want that myself14:22
anteayalet's deal with one question at a time14:22
dirkheh, we == rpm packaging core group, e.g. number80 (PTL) and me (ex-PTL)14:22
anteayathere are a number I have yet to answer14:22
anteayathen the rpm packaging core group can have a rpm-ci gerrit group and recieve verified +1 or -1 on its patches from whatever third party ci systems it wants14:23
anteayalet's address why should there be a strong relationship between voting check jobs and voting gate jobs14:24
anteayathe check pipeline runs all the jobs in isolation, each patch is tested individually14:24
anteayathe gate pipeline runs all the jobs together, each patch (in a pipeline group) is tested serially14:25
anteayaso all the jobs that vote have to pass in check for it to get into the gate14:25
anteayathen all the same jobs that vote have to run and vote again in the gate queue14:25
anteayanow for projects that don't share a pipeline, folks might not see much difference except for race conditions14:26
anteayabut for projects tested together it makes a big difference14:26
anteayadoes that make sense?14:26
anteayaI have to make myself some food now14:27
anteayawe can continue14:27
dirkanteaya: hmm, ok, I see14:28
dirkso basically it is a real problem, albeit not a very relevant one for packaging itself14:28
* dirk would love to find some time to put all of that into the zuul documentation14:28
anteayait is a real problem, what is a real problem?14:30
anteayawell it is more likely to go into the infra manual14:31
dirkit is a real problem when voting check jobs are not also voting in the gate pipeline14:31
anteayawhich is the manual for developers to use our tooling14:31
dirkand that makes a lot of sense for interdependent changes. but rpm-packaging doesn't have those very often14:31
anteayahopefully the project-config reviewers catch that and get projects to fix it14:31
dirkat least not right now14:31
dirkwe do want to get to that point though , but that is the rebuild-the-ci-in-upstream-infra task14:32
anteayaproject config reviewers review to ensure voting check jobs are also voting gate jobs14:32
anteayahttp://docs.openstack.org/infra/manual/14:32
anteayawhen you say the rebuild the ci in upstream infra task, do you mean tripleo?14:32
dirkno this is unrelated to tripleo14:33
anteayathen I don't know this task14:34
dirkbasically what was discussed in austin iirc was to have both the deb and the rpm packaging eventually produce binary packages of each git commit, e.g. a post job on the merge of a git commit14:34
anteayaokay14:34
dirkthis was achieved (mostly) for deb according to zigos mail recently14:34
dirkbut not even started for rpm14:34
anteayaokay14:34
dirkright now we have 3rd party ci's that are testing the packaging changes in the particular downstreams build environment14:34
dirkwhich also serves a purpose but a different one than a voting job in the gate pipeline would do14:35
anteayathe particular downstreams build environment, the enviroment for the third party ci?14:35
anteayathird party cis don't vote14:35
anteayathey can leave verfied +1 or -1 on patches14:36
anteayathat isn't voting14:36
anteayaplease don't say third party cis vote14:36
dirkaha, see I learn a new thing14:36
anteayathat would cause me no end of issues14:36
dirk(one more)14:36
dirkso what is voting and what is verified?14:36
anteayareviewers vote14:37
anteayahuman reviewers14:37
anteayaautomated systems verify14:37
anteayanow we do accept the words that the infra system votes14:37
anteayawith voting and non voting jobs14:38
anteayahowever14:38
anteayaif third party operators heard that a system was voting14:38
anteayathey all would want to be able to leave verified +2 or -2 on patches14:38
anteayawhich no system other than infras will be able to do14:38
anteayavoting third party ci is a politically charged phrase14:39
anteayalet's talk about third party cis leaving verfied +1 or -1 on patches14:39
anteayasince that is accurate, though a mouthful14:39
anteayaverifying patches also should be fine14:40
anteayasince it is so far a little used phrase14:40
anteayai know it is confusing14:41
anteayathird party ci systems is a challenging topic14:41
anteayaand unfortunately it doesn't get much easier with increased exposure14:41
anteayathere are too many competing interests in the space14:42
anteayaand despite time and effort alignment doens't happen14:43
anteayaat best is is kind of a constant tension14:43
anteayathat is the best case scenario14:43
anteayai feel I've lost you14:44
anteayaI'm going to make another attempt at food, my first attempt was lackluster14:45
dirkI'm trying to understand what a verified +2/-2 means in the check pipeline14:46
dirk+2/-2 means a workflow -1 right ?14:46
anteayayou won't get verified +2 or -2 in the check pipeline14:46
anteayain the check pipeline you will get a verified +1 or -1 from infra's system14:47
anteayain the gate pipeline you will get +2 (a green check mark, I think) or a -2 from infra's system14:47
anteayaand in the gate pipeline you can also get verified +1 or -1 from third party ci systems that that project has allowed to verify14:47
anteayain this view I see a green check mark beside jenkins: https://review.openstack.org/#/q/project:openstack/neutron+status:merged14:48
anteayain this view I see jenkins leaving a green +2: https://review.openstack.org/#/c/374914/14:49
anteayaalso microsoft hyperv ci has left a +1 under verified14:50
dirkanteaya: so the verified in there comes from check pipeline or from gate pipeline?14:50
anteayain the check pipeline if the infra system jobs all report success jenkins leaves a +1 verified14:51
anteayahttps://review.openstack.org/#/c/358845/14:51
anteayaif one voting job fails in the check pipeline jenkins leaves a -1 in verified14:52
anteayain the gate pipeline if the infra system jobs all report success jenkins leaves a +2 verified (and the patch should merge)14:53
anteayain the gate pipeline if one of the infra system jobs fails jenkins leaves a -2 in verified14:53
anteayaautomated systems can only report in the verified area of the gui for gerrit14:54
dirkanteaya: aha!14:54
dirkanteaya: https://review.openstack.org/#/c/379454/14:54
anteayaI hear a lightbulb14:54
dirkanteaya: so basically here the mos check should have left a verified -1 ?14:54
anteayawhat is the mos check?14:54
dirk(mos check == Fuel Packaging CI)14:54
anteayayeah lets call that fuel packaging ci14:55
anteayaI can't map mos to that and make it stick14:55
openstackgerritPaul Hummer proposed openstack/requirements: Bump pylxd global-requirements to 2.1.1  https://review.openstack.org/37952214:55
anteayafuel packaging ci has an internal problem14:55
anteayait is reporting success and verified +1 on job failures14:56
anteayathat is something to follow up on with the fuel packaging ci folks14:56
anteayaand if they don't fix it to remove verification permissions from that ci account on your repo14:57
anteayathat is the other thing with third party ci systems14:57
anteayathey require a lot of maintenance and follow up14:57
anteayajust having them have permissions doesn't mean they are using them properly14:57
anteayas/just having them/just granting them14:58
dirkanteaya: I happen to have some insights into the zuul configuration ;)14:59
anteayagreat14:59
anteayado you have what you need?14:59
anteayawe have monopolized the requirements channel for some time15:00
anteayaand happy to help clarify things15:00
anteayabut I don't want to prevent requirements work from happening15:00
dirkwhops, sorry, I thought this is a query15:01
openstackgerritPaul Hummer proposed openstack/requirements: Bump pylxd global-requirements to 2.1.1  https://review.openstack.org/37952215:01
anteayadirk: what was a query?15:01
dirkanteaya: our discussion was in a query15:01
anteayaoh I'm not saying we shouldn't discuss this, happy to do so15:02
anteayaI didn't know you didn't know what you didn't know15:02
anteayaso happy to help15:02
anteayajust apologizing to any folks who are waiting to discuss requirments things15:02
anteayadirk: have we more to discuss?15:04
dirkanteaya: -> query15:15
anteayaI'm sorry i am missing what you are trying to convey15:15
*** coolsvap has quit IRC15:22
*** openstackgerrit has quit IRC15:49
*** openstackgerrit has joined #openstack-requirements15:50
*** jpena is now known as jpena|off17:23
openstackgerritAndreas Jaeger proposed openstack/requirements: Add sphinx-testing to g-r  https://review.openstack.org/37963518:07
openstackgerritAndreas Jaeger proposed openstack/requirements: Add os-api-ref to projects.txt  https://review.openstack.org/37963618:07
*** AJaeger has joined #openstack-requirements18:11
AJaegerdirk: I pushed all the changs for os-api-ref now.18:12
AJaegerDo you want to do a new release of it as well?18:12
AJaegerdirk: https://review.openstack.org/379644 is the release - I espect we need it...18:17
dirkAJaeger: cool, thanks for pushing this18:20
dirkAJaeger: we could add to the projects.txt change then also the removal from the xfail list18:22
AJaegerdirk, where is the xfail list?18:25
dirkAJaeger: https://review.openstack.org/379203 :-)18:26
AJaegerdirk: so, once os-api-ref is released, then let's update 379203.18:29
openstackgerritAndreas Jaeger proposed openstack/requirements: Add os-api-ref to projects.txt  https://review.openstack.org/37963618:32
openstackgerritDirk Mueller proposed openstack/requirements: Fixes for check-uc jenkins job  https://review.openstack.org/37920318:33
openstackgerritAndreas Jaeger proposed openstack/requirements: Add sphinx-testing to g-r  https://review.openstack.org/37963518:42
openstackgerritAndreas Jaeger proposed openstack/requirements: Add os-api-ref to projects.txt  https://review.openstack.org/37963618:42
openstackgerritAndreas Jaeger proposed openstack/requirements: Raise constraints for os-api-ref and openstack-doc-tools  https://review.openstack.org/37970718:44
AJaegerdirk, is the above needed? ^18:44
AJaegerhope that's all ready now...18:44
dirkAJaeger: that last change should land automatically18:45
dirkAJaeger: as part of the release process with the release-review in releases/ we get a new review posted that raises upper constraints18:45
AJaegerdirk - ah, ok. Then I'll abandon...18:46
*** dolphm has left #openstack-requirements18:59
*** toabctl_ has joined #openstack-requirements19:46
*** toabctl has quit IRC19:50
*** AJaeger has quit IRC19:50
*** toabctl_ is now known as toabctl19:50
*** AJaeger has joined #openstack-requirements20:14
*** jpena|off has quit IRC21:00
*** jpena|off has joined #openstack-requirements21:02
*** anteaya has quit IRC21:05
*** anteaya has joined #openstack-requirements21:29
openstackgerritMerged openstack/requirements: Bump gabbi to 1.26.1  https://review.openstack.org/37934722:24
prometheanfirewhoa, people are talking22:30

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!