07:56:46 #startmeeting ptl_sync 07:56:47 Meeting started Tue Apr 28 07:56:46 2015 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 07:56:48 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 07:56:50 The meeting name has been set to 'ptl_sync' 07:56:51 #topic Heat 07:56:54 asalkeld: o/ 07:56:59 yo 07:57:31 are we looking at bugs that might cause another rc? 07:57:37 asalkeld: so at this point we only respin in case of a major regression, blatant security hole or stuff that can't be fixed post-release 07:57:52 https://bugs.launchpad.net/heat/+bugs?field.tag=kilo-rc-potential 07:57:53 I don't see any like that 07:58:00 Do you have anythign that meets that description there ? 07:58:19 not yet 07:58:24 https://bugs.launchpad.net/heat/+bug/1446252 sounds like a typical bug 07:58:24 Launchpad bug 1446252 in heat "Can't rollback update with nested stack in progress" [High,Fix committed] - Assigned to Zane Bitter (zaneb) 07:58:43 can land in stable/kilo post-release alright 07:58:55 yes 07:59:05 Quick look at proposed stable/kilo things 07:59:12 ok 07:59:15 https://review.openstack.org/#/q/is:open+project:openstack/heat+branch:stable/kilo,n,z 07:59:39 https://review.openstack.org/#/c/177602/ is interesting 08:00:07 looking 08:00:14 is it something we can change at any time ? 08:00:48 ttx: it is not a security issue 08:01:03 Right, this SHA1 is not actually brute-forceable enough 08:01:07 so yeah, can be done any time 08:01:31 ok, looks like you're good (which is good news, since some others are not in such good shape) 08:01:33 it's just to make a dynamic version number 08:01:47 You should work on release notes on the remaining time 08:01:57 https://wiki.openstack.org/wiki/ReleaseNotes/Kilo 08:02:07 ok will do 08:02:09 #info At RC2, no need for a RC3 at this stage 08:02:31 We need release notes completed by Thursday when we unleash the thing to the world 08:02:33 agree 08:02:47 We usually have more time to work on them but we were late this cycle 08:03:07 so the sooner the better (as you get closer you'll get edit conflicts) 08:03:15 ok 08:03:23 asalkeld: questions? 08:03:34 ttx: no all good 08:03:39 asalkeld: unless I hear from you I'll retag the RC2 as final on release day and play LP tricks 08:03:55 sounds good 08:04:04 awesome, thanks ! 08:04:15 i'll let you know if there is something critical 08:04:22 but i haven't seen any 08:04:45 asalkeld: right 08:04:58 the point is, we'll find significant bugs next week too 08:05:07 so the question at this stage is... can this wait 08:05:39 ok 08:06:00 ... yes we can 08:06:04 (wait) 09:18:26 johnthetubaguy: o/ talk to you in a minute 09:20:13 ttx: ack 09:20:20 #topic Nova 09:20:42 So we have a RC3 window opened, and it's not that easy to close with a busy gate 09:21:03 We opened it for https://bugs.launchpad.net/nova/+bug/1448075 09:21:03 Launchpad bug 1448075 in OpenStack Compute (nova) kilo "Recent compute RPC API version bump missed out on security group parts of the api" [Critical,In progress] 09:21:13 which didn't merge in master yet 09:21:55 and is failing tests again on the retry 09:22:18 so I'm a bit worried we won't be able to close this one today 09:22:34 One of the reasons I don't want to consider anything not merged in master for RC 09:23:13 oh dear, agreed with must be merged in master 09:23:20 taking a look 09:23:44 At this point on current load, that patch won't merge in the next 14 hours 09:24:07 which places the RC3 closing, at best, at 26 hours 09:24:18 not sure we can afford that 09:24:35 ah, sounds like gate has gone unstable :( 09:25:07 aweful timing, as ever with these things 09:25:20 well, I don't agree. We shouldn't depend on last week for anything 09:25:34 very true 09:25:37 RCs from last week should have been ok 09:25:53 So I blame lack of test coverage for things that are considered RC 09:26:13 thats totally fair 09:26:18 anyway, two things trying to slip in the same RC window 09:26:26 I have schedued a session for this 09:26:27 https://bugs.launchpad.net/nova/+bug/1447132 09:26:27 Launchpad bug 1447132 in OpenStack Compute (nova) kilo "nova-manage db migrate_flavor_data doesn't do instances not in instance_extra" [High,New] 09:26:41 and bug 1444745 09:26:41 bug 1444745 in OpenStack Compute (nova) "Updates RPC version alias for kilo" [High,Fix committed] https://launchpad.net/bugs/1444745 - Assigned to Alex Xu (xuhj) 09:26:57 That last one is concerning, sounds like something that should be done prerelease 09:27:05 (but hasn't been) 09:27:16 yeah, thats the one I am worried about 09:27:36 Both are actually closer to merging than 1448075 09:27:39 since they are in master already 09:27:46 basically, it causes us some upgrade issues for liberty, but I think we could work around it 09:27:46 so two options 09:28:03 1/ cancel RC3 while nothing has merged in it yet 09:28:24 2/ Do RC3 (in which case we could add both bugs in) 09:29:07 I'm leaning toward doing it with at least 1448075 and 1444745 in 09:29:15 Not totally sure about 1447132 09:29:33 That probably means closing RC3 tomorrow 09:29:41 which means it won't see any testing 09:29:49 so this fixes must be 100% safe 09:30:09 right 09:30:16 and I have a hard time understanding what those RPC things actually change 09:30:27 what's your take on the options ? 09:30:42 so my take is, its all about this one: https://bugs.launchpad.net/nova/+bug/1448075 09:30:42 Launchpad bug 1448075 in OpenStack Compute (nova) kilo "Recent compute RPC API version bump missed out on security group parts of the api" [Critical,In progress] 09:31:06 thats the only real trigger for RC3 09:31:21 ...and we just added another hour to the lead in, thanks to a patch that was top of line and -2ed 09:31:51 Now at 12:42 09:31:54 if that doesn't make it, then RC3 doesn't matter, I think 09:31:57 12 hours and 42 minutes 09:32:06 right 09:32:12 thats kinda crazy 09:32:30 so lets just revisit the other ones a second 09:32:34 and that's European morning, so as low as it will be 09:32:49 bug 1444745 09:32:49 bug 1444745 in OpenStack Compute (nova) "Updates RPC version alias for kilo" [High,Fix committed] https://launchpad.net/bugs/1444745 - Assigned to Alex Xu (xuhj) 09:32:55 that one can be backported 09:33:13 we usually forget to do that, it only really gets used by those using liberty anyway 09:33:22 not too major 09:33:30 ok 09:33:39 bug 1447132 09:33:40 bug 1447132 in OpenStack Compute (nova) kilo "nova-manage db migrate_flavor_data doesn't do instances not in instance_extra" [High,New] https://launchpad.net/bugs/1447132 09:34:01 this is a utility that helps force the database to upgrade the data 09:34:32 now we plan to force people to run that before some point in liberty, so we can drop the compat code 09:34:57 that could be first step during a liberty upgrade, if it really has to be, so we could backport that OK, possibly 09:35:12 its fairly risk free, as its changing a util not core code 09:35:30 I would like to jam that in, if we had time, but we don't, so I am OK without that I think 09:35:45 So... how about we cancel the RC3 until we know if we have a chance to make one 09:36:10 We get that other fix in master, and once it's in, we see what the lead time is for the stable/kilo patch 09:36:17 ttx: OK, so we reopen RC3 if/when bug 1448075 lands 09:36:17 bug 1448075 in OpenStack Compute (nova) kilo "Recent compute RPC API version bump missed out on security group parts of the api" [Critical,In progress] https://launchpad.net/bugs/1448075 09:36:22 yes 09:36:43 if it lands and the lead time on the gate still lets us do RC3 before Thursday 09:36:47 worst case, we give are selves some more compat code to keep during liberty, I think we can keep just those methods, but I need to check 09:37:03 ttx: yes true 09:38:55 so let's keep an eye on that patch (https://review.openstack.org/#/c/177186/) and gate queue status today 09:39:03 johnthetubaguy: we can talk at the end of the day 09:39:11 ttx: so I think the mistake we made was to be way too late trying to raise the compute RPCAPI, its a long time since we did that, and turns out we didn't really remember everything that was needed 09:39:11 In the mean time I'll kill the RC3 window 09:39:18 ttx: OK, thanks 09:39:31 ttx: so we have a first draft of Nova summit sessions in an etherpad now 09:40:00 ttx: waiting for cross project ones, and finding session leaders before we upload all of those to sched 09:41:22 ok 09:42:24 ttx: thanks sorry for the delay, stupid trains 09:42:28 johnthetubaguy: oh, we have option 3 too. Merge stuff in kilo that are not in master yet 09:42:43 Not sure that's a good idea though 09:42:47 ttx: yeah, I am not sure I like that idea 09:43:06 ok, RC3 deleted for now 09:43:51 ttx: tahnks 09:43:54 doh 09:43:55 thanks 09:45:21 johnthetubaguy: I'll let you explain the situation to the nova crowd 09:45:46 ttx: will do 09:46:02 We could ask peopel to stop approving things, but I know it's useless 09:46:55 johnthetubaguy: oh, and while you wait on gate... https://wiki.openstack.org/wiki/ReleaseNotes/Kilo 09:47:01 We need those done by tomorrow 09:47:15 the earlier, the better to avoid edit conflicts 09:48:18 ah, by tomorrow, oops, yes 09:48:44 its on the agenda for the meeting on thursday, which is no good 09:48:49 will push on that 09:49:00 yeah, at that point release will be out already 09:49:08 right 11:38:44 gate lead-in down to 4h55min 12:00:45 ttx: knock, knock ... ready when you are 12:01:27 ttx: re. the ceilometerclient stable/kilo release, I need to discuss that a bit with gordc 12:01:54 ttx: (on the agenda for our call at 13:30UTC today) 12:02:07 eglynn: ohai 12:02:19 eglynn: it's been published yesterday 12:02:25 #topic Ceilometer 12:02:59 a-ha, I see https://pypi.python.org/pypi/python-ceilometerclient/1.0.14 12:03:01 cool 12:03:01 eglynn: so at this point we only respin in case of a major regression, blatant security hole or stuff that can't be fixed post-release 12:03:10 especially with the gate being on fire 12:03:15 yep, got it 12:03:33 looking at ceilometer, you seem to be on the good side 12:03:58 yep, nothing major on the horizon 12:04:04 Even https://wiki.openstack.org/wiki/ReleaseNotes/Kilo#OpenStack_Telemetry_.28Ceilometer.29 seems to be in good shape 12:04:20 * ttx gives out the "Good Citizen" badge to Ceilometer team 12:04:43 ttx: LOL :) ... finally! 12:05:03 eglynn: that doesn't mean you get the "webscale" badge :P 12:05:20 ttx: touche! 12:05:41 But yeah, I appreciate you guys not being late :) 12:06:21 so, no remark on my side. Will retag RC2 to final on release day, unless you scream before then 12:06:32 cool, sounds good! 12:06:39 Have a nice day! 12:07:03 you too, thanks for your time! 12:08:09 SergeyLukjanov: ready when you are 12:08:22 #info No RC3 on the horizon 12:08:57 #topic Sahara 12:25:21 well, no Sergey 12:25:24 #undo 12:25:25 Removing item from minutes: 12:25:28 dhellmann: you there ? 12:25:38 ttx, hey 12:25:45 ttx, sorry for the delay 12:25:45 bah 12:25:51 #topic Sahara 12:25:53 ttx: here, but take SergeyLukjanov first I'm in the middle of a release for oslo.messaging 12:26:16 SergeyLukjanov: as I told eglynn, so at this point we only respin in case of a major regression, blatant security hole or stuff that can't be fixed post-release 12:26:26 especially with gate not behaving 12:26:30 so, no new issues in Sahara, we're ready for release 12:26:43 right, can't spot any such issues in shara backlog 12:26:55 so we are goot to go, will retag on Thursday as final 12:26:58 and I think we already fully finished testing and switched to L dev 12:27:21 SergeyLukjanov: nice. Will you be around to push the "other" shara pieces around release time ? 12:27:30 ttx, yup 12:27:48 SergeyLukjanov: I can do Sahara in the EU morning, so ping me then 12:27:55 ttx, okay 12:28:08 SergeyLukjanov: you need to work on https://wiki.openstack.org/wiki/ReleaseNotes/Kilo#OpenStack_Data_Processing_service_.28Sahara.29 12:28:22 needs ot be finalized before EOD tomorrow 12:28:32 well, not "finalized" 12:28:37 but at least be made presentable 12:28:48 so people reading them don't feel it's WIP 12:28:50 ttx, ack, we have a draft already, will finalize it tomorrow EU morning 12:28:53 ok 12:28:57 that is all 12:29:04 SergeyLukjanov: have a great day! 12:29:15 #topic Oslo 12:29:22 what's about client release from stable/kilo? 12:29:39 with fresh requirements 12:29:58 SergeyLukjanov: we can't change the requirements of stable branch clients 12:30:00 #undo 12:30:01 Removing item from minutes: 12:30:23 I mean https://github.com/openstack/python-saharaclient/compare/0.8.0...stable/kilo 12:30:24 SergeyLukjanov: is there any critical bug that would warrant a kilo point release ? Or could you just make a liberty release ? 12:30:41 we have requirements synced to stable/kilo branch of python-saharaclient 12:30:47 (Kilo release reqs) 12:30:59 do we need to have a release in this branch that includes them? 12:31:01 SergeyLukjanov: yeah, I don't think that is needed 12:31:08 no bugs 12:31:22 SergeyLukjanov: releasing an update of the library with requirements changes would require changing the minor version of the library, which would put it outside of the stable branch release series 12:31:26 you should probably revert that commit 12:31:40 that way when you have a bug fix later, you can release safely 12:31:57 dhellmann, in master we have 0.9.0 and in stable/kilo the latest one is 0.8.0 12:32:02 SergeyLukjanov: also I wouldn't backport random fixes to the lib branch, that sounds overkill 12:32:15 the old minimums without caps are compatible with the stable branch 12:32:29 okay, so, no need to release client from stable branch 12:32:38 SergeyLukjanov: confirmed 12:32:43 thx 12:32:56 #topic Oslo 12:33:05 dhellmann: so oslo.messaging release was my only topic 12:33:23 I just tagged it, and I'm looking for my sandbox with the script for generating the release note mail 12:33:47 ok, so I think we are good. Maybe we can tag stable/kilo oslo-incubator 12:34:07 and release too 12:34:08 yes, let's go ahead and do that 12:34:26 i.e. I ca push a 2015.1.0 tag and cut stable/kilo from same commit 12:34:30 you mean branch, not tag, right? 12:34:36 yeah, ok, both 12:34:38 historically there was a tag too 12:34:49 not that it changes anything, but consistency ftw 12:34:52 right, I was a little behind and meant for the stable/kilo name 12:35:08 it should be safe to do that from master, let me check 12:35:15 yep 12:36:08 I have master as 0db1430757e1a33de7d3a20f4dd7048b4374d024 and that should be OK 12:36:36 ack that is what I have 12:38:00 all set 12:38:23 I think we are kilo-done as far as Oslo is concerned 12:38:38 dhellmann: anything on your mind ? 12:39:21 we may want to talk about centralizing library releases, since the semver thing continues to be a source of confusion 12:39:36 yes, I have a working session oin RelMgt track for that 12:39:39 excellent 12:39:56 did you see lifeless' post on requirements management? 12:39:58 I'd like to take a page off oslo book and see how wer can reorg the "release team" to do that 12:40:13 dhellmann: still on my todo 12:40:14 #link https://rbtcollins.wordpress.com/2015/04/28/dealing-with-deps-in-openstack/ 12:40:36 yes, "central" would need to include several people 12:41:03 I fear that's the only way to enforce some amount of process 12:41:22 The trick is to do it without getting in the way 12:41:38 at least until we have enough community knowledge, which I fear we won't be able to build until we stop tweaking the process :-) 12:41:49 but the current process inconsistency is not really looking good anyway 12:42:10 I'm thinking of ways to automate the process, but it keeps coming back to having the signed git tags 12:42:15 and as we rely more and more on lib stuff... starts to become a clear issue 12:42:37 dhellmann: if it all boils down to having a way to review tags.... we could push for that support in Gerrit 12:43:11 I like mordred's announcements. 12:43:37 yeah, I have no idea what would be involved in making it possible for gerrit to review tags. My thought was to have a file that lists the tags, and review that, and then have a bot apply the tag after the review 12:43:40 anyway, meat for that discussion 12:43:59 but we would have to trust the bot's gpg signature, which I suppose isn't a huge issue 12:44:01 dhellmann: https://groups.google.com/forum/#!topic/repo-discuss/VV_xj6ftvoY 12:44:22 I need to learn more about gpg's interaction with git 12:44:28 oh, interesting, I'll put that on my reading list 12:44:43 well, it's just other people complaining about that feature gap 12:44:50 not really interesting reading 12:45:01 ah 12:45:17 just proving it's not completely an openstack thing 12:45:32 dhellmann: I would like very much to get someone to add tag submission to gerrit directly 12:45:50 it really just takes someone being willing to write java 12:46:04 mordred: I'll write Haskell before I'm back to Java code 12:46:04 maybe we can convince mirantis to let SergeyLukjanov do it - he likes java 12:46:29 * SergeyLukjanov reading 12:46:43 Although writing it is less painful than trying to package it, trust me. 12:47:31 dhellmann: so all that is left of https://etherpad.openstack.org/p/the-big-thaw is the external dep pinning on stable/kilo 12:47:40 dhellmann: I think we can postpone that until YVR 12:48:04 no point in doing that if we rebuild a new house of cards 12:48:07 yes, I would like to put that off as long as we can 12:48:11 right 12:48:31 mordred, I like JVM, not Java, I've been writing on pure Java about 3 or 4 years ago last time ;) 12:48:44 SergeyLukjanov: :) 12:48:53 SergeyLukjanov: but you can do ANYTHING! ;) 12:48:54 SergeyLukjanov: no no you like writing Java code and even more having on Gerrit code. 12:49:02 hacking* 12:49:05 SergeyLukjanov: I'll bet it's like riding a bike. You'll pick it up again in no time 12:49:22 dhellmann: it's more like riding a bike without a saddle 12:49:31 heh 12:49:42 heh 12:49:48 dhellmann: you can do it, but only if that's the only way to travel 12:50:07 gerrit in build using GWT and that's the main reason IMO to avoid looking in its code 12:50:24 * ttx throws up in a remote corner of the office 12:50:28 but from our PoV gerrit with tags support is very usefull 12:51:34 SergeyLukjanov: right! 13:43:57 mestery: ready when you are 13:44:02 ttx: Here 13:44:08 #topic Neutron 13:44:23 mestery: struggling a lot to close that RC3 window 13:44:31 YEs, I see that https://review.openstack.org/177789 is having issues in the check queue 13:44:34 but at least the change is in master 13:45:07 so I still hope we'll get it through 13:45:13 ++ 13:45:14 but shows the danger of late respins 13:45:21 you're at the mercy of gate weather 13:45:30 Totally agree, but on the other hand, I'm glad testing found these issues, so :) 13:45:53 well, I would even more glad if automated testing found those issues 2 months ago 13:46:02 Completely agree 13:46:21 anyway, I think we should get it in by EOD 13:46:43 and we have patches merged for RC3 already so backpedaling is not really a good option 13:46:56 so... full speed ahead! 13:47:00 Agreed 13:47:01 ++ 13:47:33 https://wiki.openstack.org/wiki/ReleaseNotes/Kilo#OpenStack_Network_Service_.28Neutron.29 looks in good shape 13:47:47 I've had people working on it, I'll go through it again and update as needed 13:48:30 mestery: if by EOD we don't have that last patch in we have two choices 13:48:58 we can delay RC3 until tomorrow (leaving little time to catch weird errors like corrupted tarballs) 13:49:06 and we can cut without that fix in 13:49:25 Well, lets just see if we can get that in 13:49:30 And not have to look at those options 13:49:41 Though I agree, those are the options should we not succeed 13:49:43 full speed ahead! 13:49:47 :) 13:50:15 suffice to say only really odd stuff will trigger a RC addition at this point 13:50:19 ttx: Just curious, how many other RC3s are in the works? Are we the only one? 13:50:36 There was a Nova one but the fix isn't in master yet so I cancelled it 13:50:37 \++ to that 13:50:43 OK 13:50:57 we may revive it if the fix finally makes it, since it's a pretty critical thing 13:51:11 but in the mean time we wait 13:51:17 OK 13:51:21 there may be a swift one 13:51:42 only news so far 13:51:54 Ack 13:52:10 * mestery puts on his zuul monitoring glasses for the day 13:52:15 the gate os completely desintegrating before my eyes 13:52:18 is* 13:52:24 I know :( 13:52:34 Lots of red on there 13:52:42 let's hope they figure it out 13:52:52 ++ 13:52:58 at least we cut on the lead time. We seem to fail faster now 13:53:07 :) 13:53:12 Positive spin I guess 13:53:16 this morning the lead time was 15 hours 13:53:21 Yikes 13:53:28 so it took 15 hours to fail 13:53:42 If it was like that now, we'd just as well spin RC3 without that last one. 13:54:07 OK, I'll be talking to you later 13:54:20 Yes, talk to you later 14:20:53 nikhil_k_: o/ 14:21:00 ttx: hi 14:21:05 #topic Glance 14:21:31 so at this point we only respin in case of a major regression, blatant security hole or stuff that can't be fixed post-release 14:21:38 especially with the gate on fire 14:21:56 nikhil_k_: do you have anything that fits that description 14:21:56 makes sense 14:22:32 I don't see anything in the kilo-rc-potential list or the proposed backports 14:27:01 nikhil_k_: in the remaining time (before tomorrow EOD) you should fill out https://wiki.openstack.org/wiki/ReleaseNotes/Kilo#OpenStack_Image_Service_.28Glance.29 14:27:14 so that it's ready for release day 14:27:39 ttx: all three? 14:27:51 Key New Features, Known Issues and Upgrade Notes ? 14:28:05 just trying to make sure if ground is covered 14:28:09 Yes. If you don't have anything to say in the latter sections, just remove them 14:28:53 If you have significant issues that won't get fixed in release, good idea to mention them in "Known Issues" 14:28:54 ttx: ok. I don't have anything else for now in Kilo. 14:28:59 ok 14:29:14 all good 14:29:20 ttx: so about the upgrade notes 14:29:36 ttx: we have tried to apply as many UpgradeImpact flags as possible. 14:29:43 Where do those flags go? 14:30:03 or you get some notification to include them if misses? 14:30:05 missed* 14:30:13 They don't automatically go anywhere. But you should be able to grep them in the git log and mention the most significant glitches 14:30:30 ok 14:30:41 I hoped some digest is already created 14:30:46 They are supposed to help in redacting that section 14:30:50 not that I know of 14:30:51 anyways, grepping is always fun 14:31:39 nikhil_k_: OK, so unless we talk again, I'll be releasing RC2 as final on Thursday. You get those release notes filled before then. 14:33:30 nikhil_k_: talk to you later! 14:33:34 thingee: around? 14:33:43 ttx: hi 14:33:45 #topic Cinder 14:34:08 bug came up with config generator. Unfortunately nova also has it 14:34:27 https://bugs.launchpad.net/cinder/+bug/1447380 14:34:27 Launchpad bug 1447380 in OpenStack Compute (nova) "wrong cinder.conf.sample generation: missing directives for keystone_authtoken (at least)" [Critical,In progress] - Assigned to John Griffith (john-griffith) 14:34:35 thingee: that was fixed in RC2 for Cinder right 14:34:50 thingee: Nova dismissed it as being broken for a long time 14:35:03 ok sorry missed that on there side. 14:35:15 (sorry, got disconnected from IRC. talk to you later) 14:35:24 as I told nikhil; at this point we only respin in case of a major regression, blatant security hole or stuff that can't be fixed post-release 14:35:35 I don't see any such things in the pipe for Cinder 14:35:49 Also the gate is on fire so we can't really respin anything at the moment 14:35:52 I've been watching the incoming bugs pretty closely and caught up. nothing important came in 14:36:07 * thingee throws a cup of water at it 14:36:14 So I'll retag RC2 as final on Thursday, unless we speak again (or you send me an email) 14:36:39