21:05:35 #startmeeting log_wg 21:05:36 Meeting started Wed Jan 27 21:05:35 2016 UTC and is due to finish in 60 minutes. The chair is rocky_g. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:05:38 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:05:40 The meeting name has been set to 'log_wg' 21:05:50 o/ 21:06:03 sorry I'm late 21:06:09 jokke_, you there? 21:06:31 yes 21:06:33 o/ 21:06:49 sorry, just got in 21:06:56 so I was bit late as well 21:07:23 OK. jokke_ meet rbradfor and vice versa 21:07:48 jokke_, hi,hi 21:08:18 and rbradfor, monty says hi! 21:08:33 monty taylor? 21:08:39 yup 21:08:41 hi rbradfor \o 21:09:09 we were discussing logging and I mentioned you. 21:09:28 yep, we have known each other, almost 10 years now. 21:09:52 Wow. So, monty was like four? 21:10:29 :P 21:10:33 #topic status updates 21:10:55 rbradfor, I know you're working on config stuff. Whaere are you and how can we help? 21:11:09 From an action item of last meeting I generated Sphinx docs of configuration options for 5 of 6 listed projects (swift doesn't play ball) to see what was possible. My demo was at http://ronaldbradford.com/tmp/openstack-opts/ 21:11:17 I took this one step further and integrated Glance into full docs which provided some surprises. https://review.openstack.org/#/c/270920/ has merged, you can see the full glory at http://docs.openstack.org/developer/glance/opts.html 21:11:41 surprises are things we should discuss in relation to configuration reference guide 21:12:27 excellent. Get to them before the endusers see them 21:12:33 rbradfor: oh that was you ;) 21:12:47 jokke_, ?? 21:13:23 rbradfor: didn't realize to relate the nick and name together, that's all 21:13:37 jokke_'s main job is glance core 21:13:43 I was looking that change but Flavio and Sabari were faster approving 21:13:55 jokke_, ok, well I didn't know that either. 21:14:10 so, thanks for that :D 21:14:35 I just picked glance at random but the interaction with Sabari was very helpful towards decisions one may make in a operator config guide 21:15:05 please elaborate.. 21:15:14 Flavio (who I did not know is PTL) gave a nice review feedback I appreciated. 21:15:28 so, some points of discussion 21:15:42 1. what do you do when a project has multiple configuration files. 21:15:55 in this case I put all files into one page. 21:16:03 this has advantages and disadvantages. 21:16:32 the main disadvantage is if you look at just one file, you don't see all possible options. 21:16:55 what you get (and it took some formatting) is specific options and a "common libraries" section 21:17:03 glance has actually at least 5 config files + policy and -paste.inis 21:17:05 which is then linked to the end of the document. 21:17:34 jokke_, I documented 5 I knew of (based on entry_points) 21:17:47 cool 21:18:06 what I learned is that you include other namespaces, via your opt/oslo-config-generator/ options (something knew I learned) 21:18:36 my first instinct was to include all namespaces into each file, but I could not actually do that, I ended up with duplicate DEFAULT sections (a later topic) 21:18:49 and, we would end up with what we are trying to avoid, repeating the same content. 21:19:12 ++ 21:19:15 so while a disadvantage you don't see all options possible, the advantage is I did not duplicate (for example logging options) 21:20:03 so, debatable if it should be one configuration file per page, and another page of "included namespaces", all linked nicely, or one page as it is now. 21:20:38 glance is actually one of the rare projects still carrying example configs in tree, so in that perspective it does not make that huge of a problem either way 21:20:52 and how it is now, could be achieved with present sphinx directives. creating one all encompassing file would have met a oslo.config code change 21:21:22 jokke_, I liked the use of the oslo-config-generator conf file, I hope other projects use that. 21:21:25 I personally like the idea of having the documentation all in single page so it's easy to search when you are crawling through those configs 21:21:37 infact, I'd propse we create a sphinx option to read these files specifically. 21:21:47 ++ 21:21:54 rbradfor: I don't, but that's different story ... Hopefully we get the reason fixed in oslo-config-generator 21:22:06 so the docs creatiion uses one conf file, and generates sample configs and matching docs. 21:22:22 jokke_, whats the fix you speak of? 21:23:02 well there is no fix yet, but the problem is that if you change something in the config options (at the code level) you get XX% different output from oslo config generator 21:23:04 as I just fixed another bug in this area, happy to hear your input while it's fresh 21:23:21 making config management absolute nightmare 21:23:28 and thus upgrade as well 21:23:56 ok, well lets come back to this, because I'd like to know what it is so I can fix it 21:24:19 FYI here, Nova is working to merge all their config files into a single one, with clear "sections" 21:24:23 IIUC Stuart McLaren opened bug against it just last week 21:24:30 haven't got to look into it yet 21:24:42 rocky_g: we are not :P 21:24:48 So, I think there will be a trend in Newton to do the same in other projects. 21:25:11 I've seen a lot of reviews on nova cleanup 21:25:23 Yup. 21:25:30 there has been push for that at least for 3 cycles now, and I'm happy to keep -2ing any proposed changes for such :P 21:25:49 speaking of sections, this is a problem I found, and I'm going to raise it in Austin, that is having logging not be in [DEFAULT] but in a [logging] section 21:25:54 same with removing the examples from tree 21:26:12 Ah. Interesting. 21:26:15 jokke_, silly to remove conf examples from code. 21:26:30 rbradfor: that's purely because it comes from oslo_log, not from the project namespace 21:26:56 I'd love to see a proposal to have a "global" config that only gets overwritten by individaul project config files. Right now, no global 21:26:58 rbradfor: and tbh. I like the fact that all the config options are now under their own section, not scattered around [DEFAULT] 21:27:17 yes, because it's a namespace, and a few others benefits 21:27:30 So, instead of "default" how about "global" with sections? 21:27:39 1 is the generation of "common libraries". all others have there own section 21:27:42 rocky_g: the problem with that is that it's great idea for devstack, but kind of pointless when you deploy the services to their dedicated machines 21:28:04 and also if you want to move to rocky_g goal of a higher order global config I see sections as being beneficial 21:28:59 I think ops would take the config files from a repo and write a distribution script to put it on the appropriate iron. 21:29:15 I think the oslo-config-generator is the right way to go if we get projects not to change those defaults what oslo__log offers 21:29:44 that would give that global default and it would be up to deployer to change if they insist 21:31:56 Well, it would be good to have a way to identify all the cross project config options separate from the project specific ones and have one place to change them... 21:32:54 rocky_g, it will take some time to get standardization first across projects. the siloed approach now has some big effects in just naming things consistently. 21:33:02 So far as I am aware, that's only oslo stuff at the moment. 21:33:10 Yeah. Don't I know. 21:33:47 rbradfor: I'm bit afraid of that naming consistently part as well ... afraid that we will get even more collisions 21:33:56 The cool thing about the generator is that it might be easier to identify conflicting options settings 21:34:50 could we do it for oslo stuff, though? And instead of a "default" or "global", just have an "oslo" section? 21:34:54 well as long as they are in different sections I don't think it cares, nor should/can 21:35:26 but when template update needs stuff like this https://twitter.com/epkuva/status/691669708756631552 naming collisions becomes really painful 21:36:15 rocky_g, just to digress a moment on sections 21:36:18 check out http://docs-draft.openstack.org/20/270920/4/check/gate-glance-docs/0e8ef17//doc/build/html/opts.html 21:36:40 which I just discovered differs from http://docs.openstack.org/developer/glance/opts.html (something else to look at) 21:36:52 you can see inthe first, in the left menu, all the sections 21:37:48 there is already inconsistency in section headings for oslo, so I doubt we will get a common concensus, and I don't think having oslo is needed? 21:38:18 oslo is only needed if there is a change from the oslo defaults 21:39:13 but, there should be links to the relavent common, included other opts 21:39:43 rbradfor: actually good example of that sorting issue I mentioned 21:40:29 jokke_, are you talking about https://bugs.launchpad.net/oslo.config/+bug/1536899 21:40:30 Launchpad bug 1536899 in oslo.config "config generator sorting has changed" [Undecided,Fix released] - Assigned to Ronald Bradford (ronaldbradford) 21:40:34 http://docs.openstack.org/developer/glance/opts.html if you look for enable_v*_api ... so we have enable_v1_api, enable_v2_api, enable_v2_registry and enable_v3_api ... but all those are scattered all over the place 21:40:49 rbradfor: I think that would be the one, yes 21:41:01 jokke_, I fixed that, it merged a few days back 21:41:11 oh brilliant!!!! 21:41:12 docs regenerated will go back to "source code" order 21:41:29 rbradfor: will that be just for the docs or for the config files as well? 21:41:42 I mena for the generated example configs 21:41:49 I'm more concerned with the draft docs and the final ones about the left menu, and section headings and formats, something is off there. 21:42:03 jokke_, it's the same code so it would fix both. 21:42:54 rbradfor: that's best news I've heard for a while ... if you don't mind, drop link to that fix to the bug and feel free to mark it fixed or invalid, which ever way you prefer ;) 21:43:14 the option group seems to clean up a lot of the toc stuff. It drops a lot of the options out of the display level 21:43:37 jokke_, https://review.openstack.org/#/c/272289/ has merged, so launchpad should mark this as closed automatically? 21:44:08 It's marked fix released 21:44:08 rbradfor: oh yeah, it actually has (Fix released) 21:44:54 makes our life so much better for the future, specially upgrades 21:45:01 jokke_, yeah, that came to Oslo attention at our meeting on monday. As I was in that code on Friday is was easy to identify, and not to much to fix 21:45:16 jokke_, so let's talk about upgrades a bit. 21:45:54 let me start with "deprecated" because I've had a ML discussion on this. I want all options marked as deprecated to have a reason, and that reason is to list "in what release it is to be removed", but that's just text. 21:46:20 it seems to me, that new options, or changed defaults also merit applicable flagging, so it would be easier to generate docs, 21:46:45 e.g. "Whats new", "Whats changed", "What's deprecated", "what's removed" sections of configuration options. 21:46:55 yeah. release note stuff at a minimum 21:46:58 that IMO would help operators in upgrades 21:47:12 I know there are release notes on this, but it's a human writing that. 21:47:37 right. auto generate would be ideal 21:47:42 if this was meta data, then the oslo-config-generator for example could have additional info at it's disposal 21:48:02 rbradfor: I'd prefer instead of having "Deprecated; will be removed in release X.Y.Z" it having "Deprecated; will be removed on first release after 1.6.2016" or so 21:48:22 that would give clear deprecation period regardless what has been released in between 21:48:51 jokke_, I could see that reason. please add to ML 21:49:13 rbradfor: will do tomorrow when I'm by my work mail again 21:49:21 jokke_, sure 21:49:35 so, back on the upgrade path. 21:50:04 what's at an operators disposal to know "whats changed/new/removed". do they simply diff sample confs between releases 21:50:48 rbradfor: that's good one 21:51:20 specially when you look that bug you fixed ... good luck finding anything from those diffs 21:51:30 before I mean 21:52:00 yes, before was death, code introduced did not retain order 21:52:00 It's one of the reasons I think it takes them 3-6 months to qualify an upgrade. They have to find all this stuff. 21:52:52 The other thing is that they can't allow their config files to be overwritten, or if they do, they then have to go back in and replace their changes 21:52:58 rocky_g: I updated our templates, the tweet I linked was how I finally got it done and even then I missed some conditions there :( 21:53:38 jokke_, here's the link to the ml thread: #link http://lists.openstack.org/pipermail/openstack-dev/2016-January/084126.html 21:53:59 the change for glance-api.conf + glance-registry.conf was give or take 3.5k lines :( 21:54:12 thanks rocky_g 21:55:15 so, the config stuff is a really big painpoint for ops 21:55:45 yes and rbradfor fixing that bug will ease it up Newton forwards 21:55:46 there was a huge thread on it when the samples were being removed. I'll see if I can find it... 21:56:44 as now folks gets version of configs from Mitaka and Newton will have just the real changes, not everything around the place 21:57:29 so, if each config option had the following attributes, (released, changed, deprecated, removal) aka the cycle (sorry names for now), it would be infinitely easy to auto-generate all the lists an operator wants between releases 21:57:56 makes sense 21:58:13 problem getting that released there at least 21:58:44 as the release version will be mainly defined at the point when the release is cut based on the changes merged 21:59:26 yes, it's some meta data, but it's a transition project during a full cycle, it's not trivial, but if a project got behind it, and then maintained it. 21:59:39 and tracking such down will be too much to just throw at the release folks 21:59:40 new options when first merged are marked with "released" 22:00:07 then if changed, deprecated they are marked. I'm sure the change/deprecate volume is way less 22:00:19 ahh ... so you did not mean that those are fields with version defined when that happened 22:00:31 tracking down should be determable via git blame or stable release tages. 22:00:43 but rather state machine of the status of the option 22:00:46 ? 22:01:06 it was just a thought now, let me describe it in an email with examples. 22:01:48 sure 22:02:06 good discussion folks ... unfortunately we're out of time 22:03:16 oops. you're right. I've got a couple of mailing list links for rbradfor, then I'll call it. 22:03:35 #link http://lists.openstack.org/pipermail/openstack-operators/2014-December/005658.html 22:03:54 #link http://lists.openstack.org/pipermail/openstack-operators/2015-January/005981.html 22:04:22 This is a thread on how operators were going to try to deal with the confg opts changes/issues/lack of sample configs 22:04:28 Thansk all! 22:04:35 #endmeeting