20:05:38 <dhellmann> #startmeeting log_wg
20:05:39 <openstack> Meeting started Wed Jun 10 20:05:38 2015 UTC and is due to finish in 60 minutes.  The chair is dhellmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:05:40 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:05:43 <openstack> The meeting name has been set to 'log_wg'
20:05:49 <dhellmann> #topic attendance
20:05:54 <jokke_> o/
20:05:56 <dhellmann> who's here for the logging group meeting?
20:06:00 <dhellmann> o/
20:06:11 <tpatil> o/
20:06:24 <dhellmann> #topic request-id spec
20:06:29 <dhellmann> tpatil: you're up!
20:06:33 <tpatil> We have reached out on the openstack -dev mailing list to get consensus on any of 3 proposed solutions.
20:06:45 <tpatil> #1 Return tuple containing headers and body from : 3 +1’s
20:06:54 <tpatil> #2 Use thread local storage to store 'x-openstack-request-id' returned from headers : 0 +1
20:07:02 <tpatil> #3 Unique request-id across OpenStack Services : 1 +1
20:07:13 <tpatil> Pros and cons of each of the above solutions is also discussed on the openstack-dev mailing list thread.
20:07:19 <dhellmann> tpatil: have you had any luck talking with the rally/osprofiler team about reusing the work they did?
20:07:25 <tpatil> #link : http://lists.openstack.org/pipermail/openstack-dev/2015-May/064842.html
20:07:45 <tpatil> dhellmann: Not yet, We have to check how does that works.
20:08:04 <tpatil> So far, Solution #1 is leading but still I don’t see any +1 on the specs
20:08:10 <dhellmann> tpatil: I think you're going to get much more traction with that
20:08:29 <dhellmann> tpatil: no, I'm -2 on that approach because it breaks backwards compatibility with existing users of those clients and we can't do that
20:09:01 <tpatil> dhellman: You are correct
20:09:30 <tpatil> dhellman:Based on your comment, we have come up with Solution #2. Please so far no have given +1 on the specs.
20:10:22 <tpatil> We need some direction from the community. We are totally committed to implement this spec for Liberty.
20:10:37 <dhellmann> ok, I have been waiting to see what the osprofiler discussion brings up, because we want to find a shared solution to this and they seem to already be doing it pretty well
20:11:16 <tpatil> Another point is, If client api bindings are used in the third party applications, then there is no way for them to know the x-openstack-request-id so they might find it difficult to resolve the issue with the service provider.
20:11:34 <tpatil> If x-openstack-request-id is made available as in Solution #1 and #2,then I think it could solve many such potential use cases.
20:11:40 <dhellmann> we can fix that with documentation, though
20:12:14 <dhellmann> yeah, changing the return type of all client API calls is really really not an option, so you need to look at the other solutions more closely
20:13:19 <tpatil> dhellman: solution #2 is next closer solution then, but so far no response on that solution.
20:13:50 <dhellmann> tpatil: as I said, I am reserving judgement until I see the update discussion osprofiler. Please talk to boris-42 and the rest of the osprofiler team about it.
20:14:40 <tpatil> dhellman: Sure , Thanks. I will understand about osprofiler till then.
20:14:49 <dhellmann> tpatil: ok, sounds good, thank you
20:15:49 <dhellmann> tpatil: you should also talk to the openstack sdk team about how best to expose request ids in the new python sdk
20:16:01 <jokke_> I'm wondering if I responded to that mail or not
20:17:11 <tpatil> dhellman: Ok, I will do that. Thanks.
20:17:55 <dhellmann> tpatil: excellent
20:17:59 <dhellmann> jokke_: ?
20:19:04 <dhellmann> #topic other pending action items
20:19:22 <dhellmann> do we have anything else we need to be working on?
20:19:48 <dhellmann> there is the message id spec to be updated, too, I wonder if we should find someone to help Rocky with those changes?
20:21:05 <jokke_> I thin we should ... I unfortunately don't have the cycles to do it atm. due to internal commitments
20:21:10 <jokke_> think
20:21:33 <dhellmann> I'm not the best advocate, since I'm not sure it's a good idea to begin with :-)
20:22:18 <jokke_> ;)
20:22:49 <jokke_> Yeah ... these two are just the big things ops are crying from us ... and I totally understand why
20:24:09 <jokke_> has there been any continuation on that log formatting change we talked about in the summit?
20:24:26 <jokke_> is there anything we could act on?
20:24:46 <dhellmann> I've been bogged down with some other work, so haven't started that, yet.
20:24:53 <dhellmann> there are a couple of parts of that
20:25:02 <dhellmann> first, we need all projects to be using oslo.context as the base for their request context
20:25:18 <dhellmann> that gives us a central place to make sure we have good values to be used in oslo.log default format strings
20:25:48 <dhellmann> so if folks have time, and want to start with something, updating projects to use oslo.context would be a good first step
20:25:48 <jokke_> ok, it would be also nice to have the projects using oslo.log as well
20:26:15 <dhellmann> yes, that's a good second step -- if a project is using neither, it may need to be updated to use both at the same time
20:26:28 <jokke_> ok, good to know ... I think there was some work done on Glance around that and I'll have a look where it's hanging
20:26:44 <jokke_> Swift is one good example IIRC they don't use oslo.log
20:26:59 <dhellmann> yeah, I don't expect them to adopt it
20:27:18 <dhellmann> let's focus on the projects that are already using the incubated version of that code, first
20:27:25 <jokke_> aa ok
20:27:33 <jokke_> we have still those around as well?
20:27:46 <dhellmann> I honestly haven't looked yet
20:28:11 <jokke_> no worries
20:28:20 <dhellmann> as far as I know, we either have projects using incubated oslo logging code, oslo.log, or they are swift.
20:28:28 <jokke_> ;)
20:28:34 <dhellmann> I don't think there are any other projects not using some form of that logging code
20:29:10 <jokke_> ok, do you know the reasoning behind swift being the special snowflake?
20:29:43 <dhellmann> history
20:29:50 <notmyname> logging predates oslo.log, oslo logging has a lot of nova-isms (at least originally) and not enough other stuff (again, originally), and logs are as much an api as anything else
20:30:01 <notmyname> or as dhellmann said, "history" ;-)
20:30:29 <dhellmann> yeah, I'm not really interested in changing that situation right now -- we have a lot of other projects that we can work with to clean up their logs
20:30:35 <dhellmann> swift's logs are good, they're just different
20:30:40 <jokke_> fair enough
20:30:46 <dhellmann> notmyname: thanks :-)
20:30:58 <notmyname> for the record, I am interested in log compatibility
20:31:05 <dhellmann> that's good to know
20:31:30 <dhellmann> it would be good to collect a summary of how swift logging is different from oslo.log, and what features may be needed to reconcile the 2
20:31:47 <notmyname> http://docs.openstack.org/developer/swift/logs.html
20:32:22 <dhellmann> k, thanks
20:32:39 <notmyname> I too would be interested in that. if you have a summary at some point, i'd love to read it
20:33:13 <dhellmann> notmyname: you might find https://blueprints.launchpad.net/oslo.log/+spec/app-agnostic-logging-parameters interesting, although it does not yet try to directly address the swift case
20:33:46 <notmyname> oh yes. the "instance" was one of my beefs with oslo.log
20:34:03 <dhellmann> yeah, we got that from nova, and we're working to get rid of it this cycle to be replaced with "resource"
20:35:29 <dhellmann> ok, let's move on
20:35:33 <dhellmann> #topic open discussion
20:35:35 <notmyname> I'm happy to take this up later. I won't jump in your meeting any longer ;-)
20:35:36 <jokke_> btw bit aside from the topic ... It's funny to follow this power battle between Swift and Nova from perspective where I haven't been involved that long that I would know all the history around
20:35:39 <dhellmann> does anyone have anything else to raise?
20:35:40 <notmyname> but good to see progress :-)
20:35:51 <dhellmann> notmyname: no, thanks for dropping by, I appreciate the feedback and references
20:35:55 <notmyname> jokke_: power battel?
20:36:49 <jokke_> notmyname: oh yeah ... lots of folks expecting everyone else doing stuff in specific way because [swift|nova] does it that specific way
20:37:25 <jokke_> it's quite funny in an annoying way
20:37:40 <notmyname> jokke_: it's mostly independent organic growth and lack of communication (on all sides) that has gotten us to where we are today
20:38:26 <notmyname> but that being said, I'm not generally displeased with the state of most things where there are differences. there is room for improvement, of course
20:39:01 <jokke_> notmyname: yeah I kind of figured that out over past few months and it is easier to filter now.
20:39:14 <notmyname> :-)
20:40:27 <jokke_> notmyname: differencies are not bad in general (otherwise we could have just one project and recycle everything) ... they become a problem when people expects the history to drive the future (or the differencies are somewhere like in logs that makes the troubleshooting nightmare)
20:41:31 <jokke_> which would be nice to get sorted ... I don't care what projects does per default but it would be nice to have easy way to uniform for those who do prefer it
20:41:32 <notmyname> when people expect their view of history to drive someone else's future ;-)
20:41:47 <notmyname> jokke_: yup. I agree with that
20:42:07 <notmyname> mostly that's a question of prioritization
20:42:13 <jokke_> notmyname: I didn't want to get to that specifics ... next good step would be start picking names as examples ;)
20:43:27 <notmyname> so to get eg logs to unify (ie the general topic of this meeting, AIUI), we need the ability in swift to configure the logged fields
20:43:51 <notmyname> and simply nobody has sat down and typed in that code
20:44:29 <notmyname> but I'd definitely support someone who wanted to, and I'd be happy to review it
20:45:24 <jokke_> notmyname: so you wouldn't see a problem bringing oslo.log in swift it the default output would look like it is now (and the would be config option to turn it looking like rest of the OS service logs) and someone would actually do the work to get that done?
20:45:25 <dhellmann> I hope to finish the projects I'm working on now in the next couple of weeks, and start working on some of this logging stuff then.
20:47:00 <jokke_> notmyname: because if that's the case I probably could try at least to allocate the cycles for M to do that (for Liberty I have already too much on my plate)
20:47:11 <notmyname> jokke_: I think a new dependency is a separate concern. suffice to say, maybe. but I'm not sure that would be necessary
20:47:20 <notmyname> does oslo.log depend on oslo.config?
20:47:30 <dhellmann> notmyname: yes
20:47:35 <notmyname> ah
20:47:47 <dhellmann> that's how deployers configure the format string
20:48:32 <notmyname> using oslo.config would be very painful for swift. unlikely to happen
20:48:52 <dhellmann> even if it's just for the log settings?
20:48:57 <jokke_> notmyname: ah, didn't think of that, but perhaps we can work around it
20:49:00 <notmyname> therefore log configuration would be done in a config format in swift's configs
20:49:55 <dhellmann> yeah, as I said, we'll get further working with some of the other projects to start
20:50:08 <jokke_> sounds like a plan
20:51:32 <jokke_> My biggest wish atm is that on the next summit we would have some concrete out of this workgroup to secure the ops collaboration with us. If we don't get anything done why should they waste their (and we ours) time on this
20:52:05 <dhellmann> yes, I think that's a valid concern
20:52:32 <dhellmann> so far the 2 proposals being championed by this group don't have a lot of traction among developers
20:53:12 <jokke_> nope ... and I'm really sad about that because those have been the two biggest painpoints the actual deployers have brought to us
20:54:18 <dhellmann> well, the request id thing just needs more work, but the message id proposal is going to cause a lot of developer pain the way it is now so I think that one needs to be very seriously rethought with that in mind
20:54:20 <jokke_> the third on same magnitude has been the log formatting
20:54:49 <dhellmann> that's going to happen just as a matter of getting everyone to use the same log library, so I'm not as worried about that
20:56:47 <dhellmann> we're about out of time, is there anything else we should try to cover this week?
20:57:30 <jokke_> I think we're pretty set ... I'll try to reach Rocky before next meeting and sync up with her to see where she needs help
20:58:29 <dhellmann> sounds good
20:58:34 <dhellmann> let's call it 2 minutes early, then
20:58:35 <dhellmann> #endmeeting