#openstack-performance log

16:00:00 <DinaBelova> #startmeeting Performance Team
16:00:03 <openstack> Meeting started Tue Apr 19 16:00:00 2016 UTC and is due to finish in 60 minutes.  The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <gokrokve> o/
16:00:07 <openstack> The meeting name has been set to 'performance_team'
16:00:17 <DinaBelova> gokrokve you're faster than bot :)
16:00:22 <gokrokve> sure thing
16:00:33 <gokrokve> faster than light :-)
16:00:55 <DinaBelova> ilyashakhat, rook?
16:00:58 <DinaBelova> gokrokve  :)
16:01:17 <ilyashakhat> o/
16:01:38 <DinaBelova> let's wait for a few moments
16:01:48 <DinaBelova> probably Ben and Alexey will join
16:03:09 <DinaBelova> ok, so let's get started
16:03:16 <DinaBelova> #topic Action Items
16:03:29 <DinaBelova> in fact we had only one action item last time on myself
16:03:38 <DinaBelova> it's regarding sharing status of  status of Keystone multinode testing
16:04:20 <DinaBelova> during last week I've installed OpenStack mitaka (using MOS 9.0 under development) on multinode env and applied all changes regarding the keystone profiling on the env
16:04:42 <DinaBelova> collected results when all DB/memcache processes are running as expected
16:04:56 <gokrokve> expected bad or good?
16:05:09 <DinaBelova> gokrokve when all of them are running :)
16:05:17 <DinaBelova> gokrokve and started to kill memcache :)
16:05:33 <DinaBelova> in fact I saw nothing bad
16:05:53 <DinaBelova> keystone - oslo.cache - dogpile - python-memcache chain seems to work ok
16:06:23 <DinaBelova> in fact although memcache server choice is a bit complex there (using hash function from key)
16:06:36 <DinaBelova> the failover mechanism is very simple
16:07:08 <DinaBelova> if the chosen server was not able to establish the connection, it's temporary moved to dead_servers list
16:07:26 <DinaBelova> and requests is processed like as it was no caching at all - via all db calls
16:07:47 <DinaBelova> the next requests to the memcached will avoid dead host and work as expected
16:07:51 <DinaBelova> gokrokve ^^
16:08:00 <rohanion> o/
16:08:02 <DinaBelova> so it seems to be very logical in fact
16:08:05 <DinaBelova> rohanion o/
16:08:31 <DinaBelova> I know MOS keystone team is planning to move from python-memcache to pylibmc as a default solution
16:08:40 <DinaBelova> in fact I'm trying to understand why
16:09:03 <DinaBelova> as I did not get clear reasons for this and dims does not know them as well :)
16:09:39 <dims> DinaBelova : i'd like to see the data. not convinced yet
16:09:47 <DinaBelova> dims indeed
16:10:24 <DinaBelova> so right now memcache connection failover mechanism is really simple, but effective (in python-memcache)
16:10:41 <DinaBelova> I did not check yet how it's working in pylibmc
16:10:44 <DinaBelova> dims ^^
16:10:59 <DinaBelova> so I most probably will take a look
16:11:22 <DinaBelova> so I'll keep this action item for the next meeting (btw, let's discuss then when to do it)
16:11:32 <dims> ack DinaBelova
16:11:39 <DinaBelova> #action DinaBelova share status of Keystone multinode testing
16:12:02 <DinaBelova> if no questions ( gokrokve ?) let's move on
16:12:18 <AugieMena> Dina, are you using the v3 api's in your Keystone testing on Mitaka?
16:12:38 <AugieMena> did you use Rally?
16:12:42 <DinaBelova> AugieMena - I'll recheck, as I uses MOS+Fuel as installer, but I suppose yes
16:12:48 <DinaBelova> AugieMena - I uses OSprofiler
16:12:59 <DinaBelova> for profiling the DB/Cache usage by Keystone
16:13:03 <AugieMena> ah, ok
16:13:23 <DinaBelova> AugieMena - I'm going through this test plan http://docs.openstack.org/developer/performance-docs/test_plans/keystone/plan.html
16:13:49 <DinaBelova> AugieMena - preliminary stats are collected from all-in-one installation here
16:13:50 <DinaBelova> http://docs.openstack.org/developer/performance-docs/test_results/keystone/index.html
16:13:58 <DinaBelova> and right now I moved to multinode
16:15:11 <DinaBelova> AugieMena - two bugs were observed https://bugs.launchpad.net/keystone/+bug/1567403 and https://bugs.launchpad.net/keystone/+bug/1567413 - both already fixed and backported to Mitaka
16:15:14 <openstack> Launchpad bug 1567403 in OpenStack Identity (keystone) mitaka "Local context cache seems to work improperly" [Medium,Fix committed] - Assigned to Morgan Fainberg (mdrnstm)
16:15:16 <openstack> Launchpad bug 1567413 in oslo.cache "Keystone fetches data from Memcache even if caching is explicitly turned off" [Undecided,Fix released] - Assigned to Morgan Fainberg (mdrnstm)
16:15:36 <AugieMena> ok, np, maybe not directly related to the topic, but we're aware of some v3 limitations in Rally and starting some work on those
16:15:50 <DinaBelova> AugieMena - very interesting
16:15:55 <DinaBelova> may you please share some details?
16:16:18 <AugieMena> yeah, I'll try to pull some info together
16:16:33 <AugieMena> can put that as an action item for me
16:16:38 <DinaBelova> ack
16:16:52 <DinaBelova> #action AugieMena share details about some v3 limitations in Rally
16:16:56 <DinaBelova> thanks :)
16:17:01 <AugieMena> sure :)
16:17:18 <DinaBelova> ok, so let's proceed
16:17:21 <DinaBelova> #topic Upcoming Summit discussion
16:17:27 <DinaBelova> again, reminder :)
16:17:53 <DinaBelova> next week (Wednesday, April 27, 11:00am-11:40am) we're having a session
16:17:57 <DinaBelova> #link https://www.openstack.org/summit/austin-2016/summit-schedule/events/7399
16:18:03 <DinaBelova> #link https://etherpad.openstack.org/p/newton-performance-team
16:18:31 <DinaBelova> I'm going to briefly go through the Mitaka results made by Performance Team
16:18:47 <DinaBelova> and discuss what is currently planned to be done in Newton
16:19:02 <DinaBelova> I hope something else will appear in the plans as a result of the session :)
16:19:18 <DinaBelova> AugieMena, rook, lezbar - are you going to attend?
16:19:34 <AugieMena> yes
16:19:42 <lezbar> Hi, yes, sure!
16:19:47 <DinaBelova> harlowja and boris-42 - I hope you guys will find some time and join as well :)
16:20:09 <DinaBelova> AugieMena, lezbar - very cool, I really wait cool and productive discussion :)
16:20:32 <DinaBelova> so please add your ideas to the etherpad if any :)
16:20:51 <lezbar> I'm looking forward to it, yes will do
16:20:54 <DinaBelova> lezbar ack
16:21:33 <DinaBelova> harlowja also promised to give us time slot for OSprofiler discussion in terms of oslo track
16:21:41 <DinaBelova> harlowja - please give us the time :)
16:21:57 * DinaBelova suspects harlowja still sleeps :)
16:22:09 <bsilverman> DinaBelova: I'll join you even though you didn't specifically ask for me :)
16:22:20 <DinaBelova> bsilverman I knew you'll join :)
16:22:22 <DinaBelova> ;)
16:22:25 <bsilverman> :)
16:22:55 <DinaBelova> ok, very cool
16:23:18 <DinaBelova> any questions regarding the summit?
16:23:27 * DinaBelova will ping harlowja separately from this meeting :)
16:24:07 <DinaBelova> it looks like nope
16:24:09 <DinaBelova> #topic Open Discussion
16:24:23 <DinaBelova> so first of all I'd like to raise question about next meeting schedule
16:24:47 <DinaBelova> obviously I propose to skip Apr 26th due to the summit
16:25:13 <DinaBelova> the question is when to have the next one
16:25:30 <DinaBelova> May 3rd is national holiday in Russia (and lots of folks are located there)
16:25:47 <rohanion> And lots of them work overtime ;)
16:25:47 <DinaBelova> and on May 10th I'll be on vacation 0_0
16:25:53 <DinaBelova> rohanion :D
16:26:10 <DinaBelova> so i wonder if May 17th will work for everyone?
16:26:19 <ilyashakhat> +1
16:26:22 <rohanion> you too, I know it :->
16:26:27 <dims> DinaBelova : is the release ready to be released? https://review.openstack.org/#/c/307600/
16:26:41 <DinaBelova> dims yeeeeees
16:26:47 <DinaBelova> and really needed :)
16:27:26 <DinaBelova> bsilverman, gokrokve, rook, lezbar, dims, AugieMena, harlowja, boris-42 - your opinion about May 17th as the next meeting date?
16:27:36 <bsilverman> +1
16:27:39 <lezbar> +1
16:28:36 <DinaBelova> #info preliminary decision to have next meeting on May 17th - do be confirmed with everyone interested
16:28:59 <AugieMena> +1
16:29:03 <DinaBelova> nice! :)
16:29:40 <DinaBelova> so that's all from my side - really looking forward for the summit :)
16:29:52 <bsilverman> me too
16:29:58 <DinaBelova> bsilverman ;)
16:30:07 <DinaBelova> 1000 nodes testing is HOT TOPIC I think :)
16:30:17 <rohanion> ouch
16:30:19 <rohanion> too hot
16:30:29 <DinaBelova> rohanion lol, don't be hurt with it :)
16:30:38 <DinaBelova> anything else to cover guys?
16:30:49 <rohanion> yes in fact
16:31:09 <DinaBelova> rohanion - go ehead please
16:31:12 <DinaBelova> ahead***
16:31:23 <rohanion> I've finished with ceilometer driver and it seems to be working fine
16:31:41 <DinaBelova> rohanion very cool!
16:31:43 <rohanion> also I've made a rough mockup of a mongodb driver
16:32:05 <rohanion> it looks like reading data from mongodb is 7 times faster than from ceilometer
16:32:18 <rohanion> 0.7 seconds vs 5.3
16:32:28 <DinaBelova> rohanion ple-e-e-ese separate to two commits - ceilometer in separated https://review.openstack.org/#/c/247005/
16:32:41 <DinaBelova> rohanion - you mean full trace generation?
16:32:45 <rohanion> yes
16:32:52 <rohanion> osprofiler trace show etc...
16:33:04 <DinaBelova> rohanion - was mongo installed on multinode? and if yes with what oprtions?
16:33:06 <DinaBelova> options*
16:33:25 <rohanion> 1 3-node replicaset
16:33:42 <rohanion> with standard options from mongodb docs
16:33:48 <DinaBelova> rohanion ack, thanks
16:33:49 <rohanion> no sharding
16:34:10 <DinaBelova> rohanion it's very promising :)
16:34:11 <rohanion> and no indexes. In future I will recommend indexing the "base_id" field
16:34:30 <DinaBelova> rohanion agree with you
16:34:53 <DinaBelova> rohanion - btw OPW interns will be announced on Apr 22nd
16:34:54 <rohanion> Hope to finish with this driver by our next meeting
16:34:57 <DinaBelova> so very soon
16:35:02 <DinaBelova> rohanion - ack
16:35:12 <rohanion> great, I'll create a reminder
16:35:13 <DinaBelova> I think you have plenty of time :)
16:35:14 <rohanion> tnx
16:35:32 <DinaBelova> rohanion - thanks for the update
16:35:55 <rohanion> you're welcome
16:36:01 <DinaBelova> ok, anything else?
16:37:01 <DinaBelova> looks like nope
16:37:19 <DinaBelova> thanks for the meeting and look forward to meet in person on the summit :)
16:37:27 <DinaBelova> have a nice day/evening
16:37:33 <DinaBelova> bye
16:37:35 <DinaBelova> #endmeeting