15:01:08 #startmeeting storyboard 15:01:09 Meeting started Wed Apr 20 15:01:08 2016 UTC and is due to finish in 60 minutes. The chair is Zara. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:13 The meeting name has been set to 'storyboard' 15:01:24 #topic Announcements 15:01:36 #info There will be no storyboard meeting next week! 15:01:44 For we will all be at the summit! 15:02:07 At the task tracking session even 15:02:15 yes! 15:02:18 #link https://www.openstack.org/summit/austin-2016/summit-schedule/events/9171?goback=1 15:02:28 it is the same day, I don't know how the time compares 15:02:33 so keep an eye on the etherpad that day 15:02:47 and the channel, probably a bit after the session 15:03:14 please bear in mind that I am completely incapable of multitasking, so if I'm talking in person, I can't read irc 15:03:27 hopefully other people on this team are a bit better at that one 15:03:44 any other annoucements? 15:03:57 _o_ 15:04:40 I think the session is 14:00 UTC 15:05:01 there was a list with utc times that I've lost 15:05:07 please ask in #storyboard for further info 15:05:26 #topic Urgent Items 15:06:08 we found the webclient hasn't been building for about a week, for storyboard.openstack.org 15:06:16 it seems to be the same problem as last time 15:06:31 ie: the workers failing to die when it's restarted 15:06:34 until the machine gets full 15:06:46 and puppet can't build because there isn't enough RAM free 15:06:56 ya 15:07:03 SotK has been investigating. 15:07:07 does anyone know where the ram usage is coming from? 15:07:16 is it time to just move storyboard.o.o to a bigger node? 15:07:21 * Zara passes the mic to SotK 15:07:28 nibalizer: probably an inundation of storyboard-worker-daemon processes 15:07:34 if not that, then I have no idea :) 15:08:21 afaik, we believe there shouldn't be anywhere near that many processes, and it's probably a bug in some bit of the storyboard worker code 15:08:29 yeah, there should be 5 15:08:47 last time we ran out of ram its because there were 100-and-something 15:09:00 so the node size is probably fine, but I'm hedging with 'probably', just in case... 15:09:02 ok 15:09:05 s/5/6/ 15:09:20 haha yes we have 119 workers 15:09:34 all who amusingly have --worker-count 5 in their proccess name 15:09:37 xD 15:09:50 i wonder if that's a clue? 15:10:08 (hopefully can fix, otherwise we'll have to kill them manually again, but that's going to get annoying. every three months, a scheduled storyboard cleanup... yeahhhh... :/) 15:10:14 this comes back to having a dev sevrer would be great because then Zara and SotK can have shell and poke 15:10:39 I've figured out why they haven't been dying properly now, next I'm trying to figure out a way to fix it 15:10:44 is it safe for me to just kill these with prejudice? 15:10:49 (yeah, SotK has some kind of setup to test it, but my understanding is that it was a pest to set up and slow) 15:11:13 nibalizer: yeah, feel free to kill them all, just restart the storyboard-workers service afterwards 15:11:13 nibalizer: I'm happy with that, SotK is it useful for you testing for them to stay alive for a bit? 15:11:18 ah, snap 15:11:20 nvm 15:11:33 excellent, thank you 15:11:51 also, thanks :) 15:12:25 cool 15:12:31 #action nibalizer to kill all the excess processes with extreme prejudice \o/ 15:12:43 yayyy 15:12:49 any other urgent items from anyone? 15:13:11 _o_ 15:13:21 guess if they were urgent, we'd know about them :) 15:13:26 #topic In-progress work 15:13:47 #info SotK has been working on worklists and boards some more (automatic boards!) 15:13:56 oh yeah 15:14:11 I was going to point to them, but webclient is behind because of aforementioned problems. 15:14:20 its possible to edit worklists from the board view now (or will be when the webclient updates) 15:14:33 this lets you make a board, then make its lanes be automatically populated 15:14:40 super secret automatic boards in master, coming soon to a storyboard.openstack.org near you 15:14:55 I'll put a link to an example on the etherpad when I can make the example :) 15:14:56 such lanes won't be manually usable though (no dragging) 15:15:30 heh, I have no idea how someone would even begin to make the rules for that. 15:16:18 well, I guess the act of removing something from a lane would have to add a very specific rule to it... anyway, /speculation 15:16:37 #info SotK has been investigating the worker mystery 15:16:57 I found that at 2amish, so that's been all today... 15:17:15 the workers sometimes get stuck here when stopping: https://git.openstack.org/cgit/openstack-infra/storyboard/tree/storyboard/plugin/event_worker.py#n156 15:18:00 I don't really know why they get stuck there, seems racy 15:18:09 still seems like you're making progress :) 15:18:24 last time I think the whole thing was a total mystery, this time you know where they're stuck 15:19:19 yeah, should get somewhere this time 15:19:39 :) do you have an idea of what possible reasons to investigate? 15:19:46 (I've seem to have fixed it by not cancelling that timer, but that seems suboptimal) 15:20:00 I can't help much but can at least be a person to think out loud to 15:21:02 beyond "figure out what would cause threading.Timer().cancel to hang", not yet 15:21:34 heh. :) if you're blocked, I can listen and/or find pictures of armadillos. 15:22:43 which should reduce stress, if nothing else 15:22:56 though I'm not a medical professional so don't cite that anywhere 15:23:19 as for me... I've mainly been reviewing this week 15:23:43 #info Zara has been reviewing, and is now preparing storyboard status report for summit 15:23:50 and anything else that needs preparing... 15:24:23 I'm hoping to wait 'til after newer webclient builds, because otherwise I'll just have to rewrite 15:24:55 so maybe 'weird misc' is a better title for my inprogress work 15:25:07 business as usual. 15:26:06 updated a couple of tiny old patches, replied to some stories, updated an ehterpad, need to update it more... eh 15:26:21 am I missing anything? 15:26:38 * SotK doesn't think so 15:26:48 there was a *lot* of review :) 15:26:58 okay, I also put betherly's tutorial on the agenda! 15:27:23 betherly, are you about? have anything to say about it? is it still on fire? 15:27:24 awesome 15:27:37 its happening for sure. i hit a pretty big road block with d3 15:28:19 :) is the roadblock the kind of thing we'd benefit from you documenting? 15:28:22 had to scrap quite a lot of the work to go with a new approach but checking out some new examples to check im going in the right direction now 15:28:43 Zara: maybe. i will try and put some stuff together :) 15:28:57 cool, I'll pester you for a note on the story if so, otherwise dw about it. :P 15:29:29 ok awesome. it will probably be after the summit now as trying to get a load of stuff ready for it 15:29:29 I'm not sure of a better place to document it round these parts, but your pain might save us some time in the future xD 15:29:48 just sad i wont have more to show for my work next week :( was hoping to have something cool to show y'all 15:30:18 aw, np, the 'don't do this it will end in flames' is actually a cool thing to show 15:30:27 even if it doesn't feel like it 15:30:43 ok cool i will do that :) 15:30:52 thank you again for helping out with this 15:31:14 and hopefully we can talk more at summit :D 15:31:24 * SotK hopes so! 15:31:41 awesome 15:32:37 #info betherly trying new approach for tutorial after hitting roadblock, Zara will pester her to document roadblock 15:32:41 hopefully that's accurate :P 15:32:50 yep thats accurate 15:32:52 thanks Zara 15:33:11 :) no discussion topics on agenda, soooo... 15:33:17 #topic Open Discussion 15:33:33 (and you're welcome, I haven't really done anything, but you're welcome anyway :P) 15:33:44 so... well, it's summit soon 15:33:50 that seems like a smart thing to discuss 15:33:51 openly 15:34:15 we've had questions about whether there are storyboard sessions at the summit 15:34:35 so, there's the community task-tracking session, but other than that, it's pretty ad-hoc 15:35:28 the vague plan in my head is 'coordinate schedules with other active storyboard developers, then if someone wants to talk, let them know where we are' 15:36:17 seems like a sensible plan 15:36:31 the 'where' might well be 'comfy seat by snack table with decent wifi' 15:36:34 that sounds good 15:36:59 im going to be mental busy especially the second half of the week bouncing around ironic/horizon/ux sessions 15:37:19 okay. my monday is mostly (maybe completely) free atm. 15:37:31 but if theres something happening or you want to meet up ping me and i will pick it up 15:37:40 (in terms of sessions I absolutely should be attending, anyway) 15:38:32 monday is pretty good for me atm too so lets make a plan to meet up then 15:38:42 so monday is the best chance for storyboard pool party-- er I mean, storyboard working session 15:38:48 and sunday night i will see you at the WOO event Zara? 15:39:07 i land at 5:30pm though following a 30 hr flight though so dont expect much sense out of me ;) 15:39:54 betherly: ack, I should be there, fine with leaving early so worry not 15:40:15 I haven't even looked at weds thru fri in detail yet 15:40:16 cool thanks Zara :) 15:40:25 i need to do some serious planning 15:41:05 * SotK too 15:41:14 tues has a few sessions, but I should still have a fair bit of time then in pinch. 15:42:18 oh, also, based on last time, a quirk of storyboard is that the most relevant sessions tend to be tues/weds 15:42:59 because of the cross-project and infra things. we actually have more flexibility at the end of the week, because we don't clash with, say, nova 15:43:03 and so on 15:44:10 though I say that without looking at the schedule yet 15:46:44 just yeah, I would assume much of monday is going to be getting to grips with the venue etc, so good for less formal meeting, tues and weds we'll be the least flexible, and the end of the week will be better for more focused hacking. 15:47:03 but that's a very vague forecast and doesn't take into acct everyone being sleepy by the end of the week 15:52:08 any other things to discuss? 15:52:11 _o_ 15:52:21 I *should* have worked out the schedule a bit more before this, so sorry about that 15:57:01 so, some sort of tentative storyboard monday meet planned, task tracking session weds morning, other storyboard summit things tbc 15:57:50 seems good to me 15:59:16 we'll keep people posted in #storyboard, the one-stop-shop for all your storyboard needs! 15:59:35 #endmeeting