17:00:05 #startmeeting ironic 17:00:06 Meeting started Mon Oct 19 17:00:05 2015 UTC and is due to finish in 60 minutes. The chair is jroll. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:08 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:10 hi 17:00:11 The meeting name has been set to 'ironic' 17:00:13 hi all, who's here for the ironic meeting? 17:00:17 o/ 17:00:28 o/ 17:00:31 o/ 17:00:33 I am o/ 17:00:44 o/ 17:00:47 o/ 17:01:14 #topic Announcements 17:01:17 o/ 17:01:28 the summit is next week \o/ 17:01:28 o/ 17:01:29 hiya folks 17:01:36 design summit schedule is here: 17:01:38 #link http://mitakadesignsummit.sched.org/overview/type/ironic 17:01:41 oh so much packing to do 17:02:03 I've also created a general etherpad for ironic folks during the summit; links to other pads, dinner info, etc. that is here: 17:02:05 #link https://etherpad.openstack.org/p/summit-mitaka-ironic 17:02:35 the openstack liberty release went out last week and includes ironic 4.2.0 17:02:43 o/ 17:02:46 we will release 4.2.1 with a couple fixes tomorrow 17:02:47 we've got cinder session which will be interesting for ironic too 17:03:01 and last but not least, our gate is broken again \o/ 17:03:08 :( 17:03:12 e0ne: got links to those specifically? 17:03:22 does anyone else have announcements or reminders? 17:03:30 i'm trying to find 17:03:43 jroll: we're just waiting on the revert to land to fix the gate... its already beeen +a'd 17:03:45 i'll add it to the etherpad 17:03:46 ? 17:04:04 NobodyCam: afaik yes 17:04:09 e0ne: ty much 17:04:09 :) 17:04:24 \o 17:04:33 https://www.openstack.org/summit/tokyo-2015/schedule/design-summit 17:04:45 https://mitakadesignsummit.sched.org/event/7527f39cf51d1eed383d6d890571d589?iframe=yes&w=&sidebar=yes&bg=no#?iframe=yes&w=i:100;&sidebar=yes&bg=no 17:04:50 NobodyCam, jroll, neutron fix was merged, waiting for confirmation 17:05:03 dtantsur: awesome ++ 17:05:03 aka I've rechecked a couple of things 17:05:10 ooops, ^^ it's not my session 17:05:12 any confirmation on when the 4.3 will be cut? 17:05:14 cool, thanks dtantsur 17:05:21 gabriel: not at this time, no 17:05:58 if there's nothing else... 17:06:03 #topic subteam status reports 17:06:10 as always, these are located on our whiteboard: 17:06:16 #link https://etherpad.openstack.org/p/IronicWhiteBoard 17:06:21 I'll give folks a few minutes to review 17:06:30 jroll: might it be this week, or are you more for after the summit? 17:06:44 there are 2 critical bugs? 17:06:47 gabriel: after summit, thinking sooner than later 17:06:51 gabriel: I think there is an item on the agenda to discuss that 17:07:01 OK 17:07:05 thanks folks. 17:07:10 Sorry I'm late. Had to reconfigure IRC client to use new ZNC host :( 17:07:20 or not... 17:07:25 what does critical mean. ironic doesn't work? 17:07:35 rloo: yeah, one is gate break, and one has a patch up 17:07:49 the latter is probably 'high' but it's really nasty if it's there 17:08:08 jroll: just looked. 17:08:12 #link https://bugs.launchpad.net/ironic/+bug/1506657 17:08:12 Launchpad bug 1506657 in Ironic "Ironic conductor Hash ring reset needs to be independent of sync_local_state periodic task" [Critical,In progress] - Assigned to Zhenguo Niu (niu-zglinux) 17:08:52 I would say it's High, but yeah, pretty nasty 17:08:52 I've read 'release' instead of 'meeting'. I think I'm a little anxious... :P 17:10:06 jroll: fwiw, I don't consider that one Critical 17:10:18 devananda: sure, but it got people writing patches :) 17:10:34 I agree 17:10:35 :p 17:10:37 heh 17:10:42 betherly: krotscheck: who is the question in the webclient section directed at? 17:10:48 * rloo changes all bugs to critical 17:10:52 rloo++ 17:10:52 devananda: rloo I changed it to high 17:11:07 thx jroll 17:11:11 rloo: lol :) 17:11:40 betherly: I'm fine with a horizon panel short term, having any upstream web UI is better than nothing 17:11:46 jroll: That's a betherly question 17:11:52 jroll: apologies to not be specific on that. generally yourself/cores of ironic/anyone with strong opinions on the matter 17:11:54 jroll: ++ 17:12:09 jroll: ok great thanks 17:12:24 does anyone disagree there? 17:12:28 betherly: does it mean we/you will have to support both versions later? 17:12:32 * jroll wants to keep moving, lots to talk about today 17:13:57 betherly: rloo: maybe take this to the mailing list? :) 17:13:57 rloo: the long term plan is to have a replacement that can replace the horizon panel ultimately but until that is sustainable a horizon panel is required. so longer term no, short term yes 17:14:16 betherly: krotscheck: any sense how "long term" that is? 17:14:23 i mean, are we talking more than one full cycle? 17:14:50 devananda: Well, horizon has halted all angular development until they can figure out what the "standards" are. 17:15:13 so yes, could be more than one cycle. 17:15:22 krotscheck: that's ... great 17:15:28 indeed devananda 17:15:44 devananda: My best guess? they'll argue about it for a couple of weeks until their internal PM's start asking about shipping features, and then everything will fizzle out and nothing will come of it. 17:15:56 krotscheck: yup 17:16:09 krotscheck: end result of that will be they continue on with the current architecture 17:16:12 can we take this one to the list? 17:16:33 as I suspect it won't be an easy answer, and we're at 10m cap on subteam topics 17:16:36 jroll: sure, take it to the list 17:16:50 betherly: could you please bring this to the mailing list? thanks in advance! :) 17:16:50 jroll: I'd love to get feedback for krotscheck / betherly re: whether we (the ironic community) want them to continue on a horizon panel or an angular panel 17:17:11 devananda: I would too, I'd also like to get to the other 5 topics :/ 17:17:16 *nod* 17:17:24 * jroll timeboxes 17:17:34 * krotscheck has an obvious and easily guessed stance on that. 17:17:36 #topic When is the next meeting? 17:17:44 rloo: that's you 17:17:51 that's my question :) 17:17:53 heh 17:18:00 so clearly we should cancel next week's meeting 17:18:03 i'm thinking we don't want a meeting next monday or the monday after? 17:18:10 rloo: ++ 17:18:11 do folks want to cancel the one after? 17:18:15 we usually skip the monday meeting right after summit because travel & recovery 17:18:17 I think we should 17:18:24 anyone disagree? 17:18:31 * jroll counts down from 30 17:18:35 lazy consensus says .... 17:18:48 nope 17:18:52 I'm good with canceling 17:18:53 * krotscheck is not travelling, so he'll be fresh and ready to go! 17:18:56 I'm on PTO a week after 17:19:06 cool, easy one. 17:19:08 #agreed next meeting will be nov 9 17:19:09 that decides it. no dtantsur, no meeting! 17:19:14 ^_^ 17:19:17 :) 17:19:24 #topic Review Lenovo driver spec proposal 17:19:26 #link https://review.openstack.org/#/c/208319/ 17:19:32 huangkai: that's you 17:19:34 Thanks 17:19:39 I'm not sure what the question here is 17:19:50 The driver ran for several review 17:20:01 no more coming comments tiil now 17:20:15 huangkai: there's a -1. was it addressed? 17:20:23 that's a fresh -1 fwiw 17:20:24 Hope everyone can look at it and rate if that's not too hard 17:20:35 huangkai: ok, is there something to be discussed in this meeting? 17:20:41 meeting topics shouldn't be used to ask for reviews 17:20:47 Not really, just a reminder 17:21:00 Pending for quite a while 17:21:03 ok, let's save that for open discussion next time 17:21:04 thanks 17:21:08 #topic Ironic and cinder integration proposal 17:21:12 ok 17:21:14 #link https://review.openstack.org/223217 17:21:16 e0ne: you're up 17:21:51 jroll: thanks 17:22:17 I've just want to inform ironic community on our (Cinder) initiative 17:22:41 we're working on attachement w/o nova blueprint 17:22:54 all API is already present in cinder 17:23:04 excellent :D 17:23:10 we want to inplement lightweight client for it 17:23:20 and we need your feedback 17:23:55 we'll have some session on summit for it, unfortunatlely, I can't find a link now:( 17:24:26 great 17:24:35 e0ne: attachment inside a tenant instance? 17:24:37 #link https://review.openstack.org/#/c/224124/ - here is a spec 17:24:41 e0ne: can you add the link to the sessions etherpad when you find it 17:24:57 nm 17:24:58 sambetts: attachment to any instance or a baremetal host 17:24:59 :) 17:25:03 AIUI, tl;dr for this is run something like "cinder volume-attach" from within an instance, it will make the correct API calls and such and attach it 17:25:10 NobodyCam: sure, I'll do it 17:25:34 wow, that's sounds awesome work 17:25:35 I'd love some ironic eyes on that, it's an important one. previously the APIs used here were "private" so to speak 17:25:45 jroll: we don't want to do it in a cinderclient 17:25:53 jroll: to not confuse anyone 17:25:55 that sounds like awesome work* 17:26:04 e0ne: oh, neat, I was judging by the POC patch :) 17:26:04 jroll: in the proposed nova spec, however we took help from nova in "attaching" the cinder volume 17:26:05 jroll: it would be a python-brickclient 17:26:17 jroll: it's an old PoC 17:26:20 yeah, that looks really cool :D 17:26:21 rameshg87: that spec is boot from volume though, right? 17:26:26 people even have code for changes with ironic virt driver in review 17:26:28 here is a new one: https://github.com/e0ne/python-brickclient 17:26:30 jroll: yes 17:26:51 rameshg87: yeah, this is just for attaching an existing volume to an existing instance 17:26:52 fyi, https://review.openstack.org/#/c/215385/ 17:26:58 jroll: oh okay :) 17:27:41 does anyone have questions/comments/concerns here? 17:27:56 e0ne: sounds awesome for ironic and more generally useful to anyone using cinder volumes, too. thanks! 17:28:06 thanks for feedback! 17:28:25 please, review spec and comment it. any feedback is needed and welcome 17:28:50 you (ironic) will be a primary user of this feature, so we need to colloborate with each other 17:29:03 ++ 17:29:22 alright, I'm gonna move on now, thank you much e0ne :D 17:29:30 #topic Brainstorm on mid-cycle locations and possible dates 17:29:32 jroll: thanks 17:29:38 jlvillal: this is you, 10 minutes enough time? 17:29:47 jroll: I would think so. 17:29:54 cool 17:29:57 gopher it 17:29:59 Just wondering if people had ideas on locations and dates for the mid-cycle. 17:30:05 I don't think we need to come to a decision. 17:30:19 But it would be good to get some ideas to propose. 17:30:28 I hear Palm Springs is nice that time of year 17:30:37 lol 17:30:39 so, we talked about north america vs europe last week a bit 17:30:41 * NobodyCam *ducks* 17:30:42 Seems like there was some interest in Europe. And then of course are typical USA location. 17:30:45 please, somewhere slightly warm 17:30:55 I hear anywhere that I do not have to do the planning is nice that time of year *wg* 17:31:02 heh 17:31:15 so what do folks think is the best way to do this, some sort of poll? 17:31:20 Maybe we should think about a good date frame first? 17:31:29 do we want it state-side or ?? 17:31:34 my open questions are: 1) what dates work for people, and 2) US vs europe vs either 17:31:35 Or location first? 17:31:45 last year, I held the "winter" midcycle in europe -- that got rather poor attendance from US companies. not that past experience is a good predictor of future ... 17:31:55 how about asia? 17:32:03 ++ 17:32:05 for me: 1. not overlapping with FOSDEM, 2. not US/UK 17:32:05 did anyone ask the Asians? 17:32:09 jroll: I'd recommend talking with potential host companies 17:32:36 * jroll hears NobodyCam has plenty of room to host in his house if we do palm springs 17:32:45 lol 17:32:53 lol 17:33:03 yep all 948 sq ft of it 17:33:05 might be able to host in toronto canada but not going to check unless we are serious :) 17:33:22 * dtantsur has an open Canada visa till spring 17:33:34 I could see an Asian mid-cycle in addition to a non-Asian one. I think attendance from outside of Asia would be sparse. 17:33:35 it is cold in toronto in jan-feb. 17:33:48 I did hear that HP Galway was looking to host.. but I have nothing concrete on that 17:34:06 where it is held depends a fair bit on what company wants to sponsor hosting it ... 17:34:06 NobodyCam: but... that will be cold :( 17:34:15 I *could* talk to my overlord, but cold it would be 17:34:22 though those conversations may be better done offline 17:34:23 yea 17:34:33 right 17:34:43 so I'm going to start a couple of polls here: 17:34:45 #link https://etherpad.openstack.org/p/ironic-mitaka-midcycle 17:34:52 maybe we can get a date range here? 17:34:53 and we can go from there after the summit? 17:34:54 and last time I took a poll of "who wants it in europe" the response was much mor epositive than the actual travel funding 17:34:55 yeah 17:34:58 if you guys want CA, i suspect yahoo sunnyvale can host it again :) 17:35:18 rloo: that is a great location 17:35:24 :) 17:35:26 yeah, taht is an awesome location 17:35:31 s/taht/that 17:35:33 NobodyCam: just no free cookies this time! 17:35:39 :( 17:35:41 lol 17:35:43 rloo: as long as there's free coffee :) 17:35:47 *frowns* 17:35:54 jroll: that, we have 17:35:58 * BadCub like cookies 17:36:16 I'll circulate the poll on the mailing list, talk with some folks about venues, and go from there, sound good? 17:36:18 I've done these things before at IBM Austin, but Austin mid then Austin for summit = meh 17:36:25 jroll: Sounds good to me. 17:36:42 jroll: ++ 17:36:43 ++ 17:36:47 ++ 17:36:48 dates first then location 17:36:49 jroll: also, do we want to colocate midcycle with another project? 17:36:52 krtaylor: yeah, agree, that's why I'm not pushing for rackspace venue either 17:36:55 rloo: good question :) 17:37:06 I'll see what nova is doing and make a note, that may be valuable 17:37:07 FYI, Jan 21 is the M2 milestone 17:37:14 neutron and cinder are candidates 17:37:17 ditto for neutron/cinder 17:37:22 ++ to colocating 17:37:47 Nova is: http://doodle.com/poll/88sbzgcv28rww2n3 17:37:50 jroll: I've had some discussions with the foundation staff previously about them helping to coordinate / arrange a multi-project midcycle 17:37:58 two of ten minutes left 17:38:08 devananda: interesting 17:38:29 thanks jlvillal 17:38:32 * dtantsur confirms that gate is fixed 17:38:38 \o/ 17:38:42 dtantsur: w00t 17:38:47 \o/ 17:38:49 :D 17:38:52 alright, I'll shoot a mail out later today, everyone good with moving on? 17:38:59 jroll: + 17:39:01 Works for me. Thanks. 17:39:13 cool, thanks for bringing it up jlvillal 17:39:18 :) 17:39:43 #topic Driver documentation 17:39:46 #link http://lists.openstack.org/pipermail/openstack-dev/2015-October/077132.html 17:39:50 * rameshg87 is here half sleepy 17:39:51 rameshg87: you have the floor 17:39:53 :) 17:39:57 i don't want to take too much time. if people prefer discussing this outside meeting, i have no objections moving on. 17:40:01 here are the concerns re-iterated: 17:40:06 1) documentation patches take longer time to merge, obviously treated just like other code patches. 17:40:11 2) don't want to take time from other reviewers for hardware specific documentation 17:40:14 3) it is almost impossible to update the documentation on stable branches 17:40:29 stable branches information is important to be updated because it is kilo documentation that people ask right now (in beginning of mitaka) 17:40:38 * rameshg87 is done with copy-paste :) 17:40:58 thoughts anyone ? feel free to reply to mailing list 17:41:07 so my biggest question: one concern is docs take "too long" to merge. what timeframe is expected/desired? 17:41:26 (this was also asked in the thread) 17:41:28 probably 1 day or two unless there are pending concerns from reviewers ? 17:41:31 * rloo already replied to mailing list 17:41:38 I just saw the reply just before the meeting 17:41:39 sorry 17:41:47 rameshg87: no worries :) 17:42:04 rameshg87: what's the need for a one day turnaround time? 17:42:17 rameshg87: don't the doc patches typically take 1-2 days unless there are pending concerns? 17:42:49 jroll: no strict requirements, but why keep it open for so long :) 17:42:59 rameshg87: it just seems to me that there are usually pending concerns 17:43:00 rameshg87: I mean, that goes for all patches then right? 17:43:05 your proposal still requires at least one core... 17:43:34 rloo: not necessarily. 17:43:40 when you have one +2, gaining the 2nd is not that hard. I personally try to prioritize patches with good feedback 17:43:40 I'm 100% against the wiki, and mostly against a separate repo. multiple sources of truth for docs makes for a horrible user experience 17:44:06 I'd also love to see examples of problematic patches that are driving this desire 17:44:06 how do other projects handle vendor driver documentation? wouldn't putting it in the wiki instead fix this? 17:44:09 jroll: rloo: I agree talking about timeframe when it should be merged is a hard thing 17:44:34 rloo: jroll: another thing is stable branches (which I missed in the email) 17:44:47 krtaylor: i'm for wiki so vendors can document what they want and as long as we aren't responsible for their documentation 17:44:49 dtantsur: yes, thanks, I think most people do that. 17:44:52 krtaylor: 1) anyone can edit the wiki, someone could come replace the entire doc with "throw your ilo server away and buy xxx instead"; 2) no quality control, at all; 3) two sources of truth 17:45:01 rameshg87: the stable branch thing is a different issue 17:45:06 rameshg87: the stable thing should be taken up with stable maintenance folks 17:45:18 jroll, true, but it doesn't happen in practice 17:45:32 krtaylor: 2 and 3 do 17:45:56 agreed, but then vendors control the docs 17:46:00 jroll: wrt 2, that's what they're basically asking for. not much/no quality control except by themselves. 17:46:29 rloo: right, I think that's going to make for a poor user experience, in addition to multiple sources of truth 17:46:34 sources are a good point, but maybe that can be a link pointing to the specific driver docs? 17:46:47 FWIW I see a lot of useful comments on the vendor docs each time, so I believe moving it to wiki will lower the quality 17:46:50 jroll: wanted to avoid the two sources of truth as well. 17:47:04 dtantsur++ 17:47:05 jroll: I tend to believe in the points you jaut made :) 17:47:15 jroll: we already have multiple sources of documentation. i agree, there shouldn't be multiple sources of Truth, but I think that can be dealt with. 17:47:37 I do want to investigate what other projects do, and perhaps the single +2 thing might be fine 17:47:42 dtantsur: why do you think moving it to wiki will lower the quality? 17:47:43 rloo: O_o where? 17:47:47 jroll, thechallenge that vendor driver owner face was taht the vendor driver features usally are ranked lower priority and hence code will merge late in the release cycle. It leads very little time for author to write doc and for review process. 17:48:09 rloo, because "I see a lot of useful comments on the vendor docs each time" 17:48:13 if one source, then I think it should stay with the same review process 17:48:13 jroll: that was a general comment. eg, we have our intree docs, the configs are described via the docs folks. 17:48:22 wanyen has a point. how can we fast-track the vendor docs once their driver (changes) have landed? 17:48:31 Is being less stringent on vendor docs a no-go? Requiring only one core to approve? 17:48:37 so for Liberty iLO doc is not even merged to the liberty stable branch 17:48:42 devananda: wanyen: honestly, we shouldn't be landing code like that without docs. 17:48:54 I think that's the real fix to wanyen's problem statement. 17:49:01 jroll, ++ 17:49:01 jroll has a good answer to this question: require docs with patches 17:49:06 jroll: I often see the docs in a separate patch 17:49:14 it also simplifies life for reviewers, to be honest 17:49:17 devananda: yeah, I didn't say we're doing it right today 17:49:27 jroll: and I'm fine with that. different folks are better at reviewing docs vs. code some times 17:49:28 maybe some type of tag added to the comment message "Doc update for Landed code changes"? 17:49:44 NobodyCam: I see that all the time. doesn't seem to help 17:49:57 :/ 17:49:59 jroll: i disagree with having one patch that includes doc + code. 17:50:07 jroll: we don't always have one patch that implements a feature. 17:50:13 rloo: why? code isn't very usable without docs 17:50:20 in that case, put it at the end of the chain 17:50:24 jroll: I agree with rloo 17:50:25 jroll: and sometimes docs are written/need iterations. most time. 17:50:25 jroll: it often is 17:50:27 jroll: it could be last patch in the chain 17:50:37 jroll: preventing code from landing means more rebasing etc. 17:50:43 I tend to think if it's in the chain it will get noticed 17:50:48 jroll: that is exactly what i've seen driver authors do during the last cycle 17:50:52 jroll: at the end of the chain works for me. 17:51:07 jroll: but that's basically what they are doing. adding doc patches at the end of the chain. 17:51:20 devananda: have you seen success with that approch? 17:51:41 rloo: okay, maybe I was mistaken then 17:51:45 jroll: honestly, if i see doc patches that are ok, i +2. maybe i am being too picky but if folks want their patches to land quicker, maybe they should spend more time or whatever to get it up to a better level. 17:51:50 I still want to see examples of problematic patches 17:52:00 honestly, in my experience, ironic reviews/lands doc patches quickly compared with other projects 17:52:07 jroll, some features are inter-related and require some re-org to the overall doc so it's going to be hard to do code+patch 17:52:27 just fyi: 8 minutes left 17:52:36 wanyen, maybe we should not demand it really... but if you want it to get in faster, having one patch is often better 17:53:30 dtantsur: i disagree. unless the doc changes are well written. otherwise, it'll delay the entire patch. 17:53:41 rloo++ 17:53:44 dtantsur: and if the doc changes are well written, it doesn't matter if they are in the same patch as code, or in a separate patch. 17:53:50 2 delayed patches take more time, than 1 delayed 17:54:05 you're assuming that doc and code -1's will go sequentially 17:54:16 this only works for inital doc/code lands too, for doc updates/interations we'd still have the issue 17:54:26 dtantsur: no. but i've seen in the past where someone has doc + code, and the code is ready to go but the doc isn't. 17:54:42 dtantsur: so we iterate over the doc, and the code then has to be rebased in the meantime. 17:55:47 I think verndors have enough incentives to have good documentation. Putting that on one pach just because there should be docs with the respective code does not help having better documentation. 17:56:02 okay, may be coming back to some resolution 17:56:18 rloo, " the code is ready to go but the doc isn't" why do you think it happens often? I usually see the opposite.. 17:56:22 no, but it does help us to ship docs with the code 17:56:33 dtantsur: cuz developers can't write very well? 17:56:35 I feel like we should take this back to the mailing list 17:56:46 jroll: I agree. back to mailing list please. 17:56:47 the more locks for having code in, the harder will be to contribute, the less we are going to have contributions in general 17:57:00 code isn't done until there are docs for it. 17:57:14 I wonder how soon we'll end up with separate patches for unit tests.. 17:57:15 okay, please share your thoughts on mailing list 17:57:19 wanyen: rameshg87: can you please provide examples of patches where you have experienced this problem, on the mailing list? 17:57:33 jroll: ack 17:57:34 actually, I'll reply with that question 17:57:40 thanks 17:57:47 #topic Open Discussion 17:57:50 two minutes :P 17:57:57 dtantsur: ? do you think we should ask for bits of doc related to each individual patch for a feature? 17:57:57 keep talking about docs or anything else 17:58:05 jroll: I am having problem that iLo dirver doc is still under review and not merge to Liberty branch 17:58:11 no results from my searches for a venue for a dinner 17:58:17 wanyen: the stable branch thing is a separate topic 17:58:21 jroll: I agree with you on that. I just think that growing the community is more important than having the right docs right there right then. 17:58:27 only one reminder: spare some time for Lenovo driver spec with comments (especially cores), thanks :p 17:58:27 rloo, that would be awesome, to be honest.. 17:58:43 jroll: but it ties to the entire vedor's driver review process 17:58:53 I eant doc review process 17:58:55 wanyen, most of the folks do not have rights to approve stable/* changes 17:58:59 wanyen: wrt the ilo driver doc. i had asked whether we wanted that patch in stable/liberty or whether we were ok having it on master, and no one seemed to think it is a high priority to get into stable/liberty. 17:59:05 but yeah, it's the whole different topic 17:59:07 wanyen: "no doc changes" is a stable branch rule which I do not control 18:00:02 really good meeting ... thanks to everyone 18:00:06 ^ 18:00:08 jroll: we could have tried to get that ilo doc into liberty but we don't typically do that for docs cuz we didn't know we have docs for each release. 18:00:08 #endmeeting