| tbarron | bswartz: (1) implement https://github.com/openstack/manila-specs/blob/master/specs/pike/ensure-share.rst, so driver can choose to generate a new hash if it needs a refresh by e.g. returning {'access_list_epoch: <new value>} to get_backend_info() call from the manager. | 00:17 |
|---|---|---|
| *** gaurangt has quit IRC | 00:20 | |
| tbarron | (2) Add a new enum value for share_instance['access_rules_status'], say 'NEEDS_REFRESH', such that (a) driver can return {<share-insnace id: {'access_rules_status': NEEDS_REFRESH}, ...} to the manager's ensure_shares() call, and | 00:21 |
| *** gaurangt has joined #openstack-manila | 00:22 | |
| tbarron | (b) the manager can modify its logic around share_instance['access_rules_status'] to send a full refresh of rules in the case of NEEDS_REFRESH instead of only the current condition where | 00:22 |
| tbarron | access rules for a share are sent in case share_instance['access_rules_status'] != constants.STATUS_ACTIVE and there are pending APPLYIING or DENYING access rules. | 00:24 |
| tbarron | gouthamr: please sanity check the above at your leisure ^^^ | 00:25 |
| tbarron | bswartz: I would in parallel be working on a backend shared state solution for cephfs/nfs-ganesha driver with the state persisted outside manila so that we shouldn't have to invoke such a refresh simply because we are doing a failover. | 00:27 |
| tbarron | bswartz: but we'd want the refresh as a fallback path and depending on timing of various deliverables might use it as a temporary stopgap. | 00:28 |
| *** hongbin has quit IRC | 00:44 | |
| *** jmlowe has joined #openstack-manila | 00:51 | |
| *** markstur has quit IRC | 00:53 | |
| *** markstur has joined #openstack-manila | 00:54 | |
| *** markstur has quit IRC | 00:59 | |
| *** markstur has joined #openstack-manila | 01:46 | |
| *** markstur has quit IRC | 01:51 | |
| *** kaisers_ has joined #openstack-manila | 01:53 | |
| *** kaisers has quit IRC | 01:56 | |
| openstackgerrit | JohnnyChou proposed openstack/manila master: Add Infortrend Manila Driver https://review.openstack.org/465846 | 02:24 |
| *** gaurangt_ has joined #openstack-manila | 03:41 | |
| *** gaurangt has quit IRC | 03:42 | |
| *** gaurangt_ has quit IRC | 03:50 | |
| *** gaurangt has joined #openstack-manila | 03:54 | |
| *** arnewiebalck_ has joined #openstack-manila | 04:12 | |
| *** kaisers_ has quit IRC | 04:37 | |
| *** pcaruana has joined #openstack-manila | 04:43 | |
| *** gouthamr has quit IRC | 04:55 | |
| *** gaurangt has quit IRC | 04:59 | |
| *** gaurangt has joined #openstack-manila | 05:00 | |
| *** arnewiebalck_ has quit IRC | 05:06 | |
| *** arnewiebalck_ has joined #openstack-manila | 05:07 | |
| *** arnewiebalck_ has quit IRC | 05:22 | |
| *** kaisers has joined #openstack-manila | 05:38 | |
| *** kaisers has quit IRC | 05:44 | |
| *** pcaruana has quit IRC | 05:49 | |
| *** pcaruana has joined #openstack-manila | 06:02 | |
| *** jprovazn has joined #openstack-manila | 06:18 | |
| *** gaurangt_ has joined #openstack-manila | 06:25 | |
| *** gaurangt has quit IRC | 06:25 | |
| *** kaisers has joined #openstack-manila | 06:41 | |
| *** gaurangt has joined #openstack-manila | 07:06 | |
| *** gaurangt_ has quit IRC | 07:06 | |
| *** td has quit IRC | 07:07 | |
| *** markstur has joined #openstack-manila | 07:12 | |
| *** gaurangt_ has joined #openstack-manila | 07:14 | |
| *** gaurangt has quit IRC | 07:15 | |
| *** kaisers has quit IRC | 07:15 | |
| *** markstur has quit IRC | 07:17 | |
| *** gaurangt has joined #openstack-manila | 07:18 | |
| *** gaurangt_ has quit IRC | 07:18 | |
| *** td has joined #openstack-manila | 07:19 | |
| *** markstur has joined #openstack-manila | 07:26 | |
| *** dsariel has joined #openstack-manila | 07:26 | |
| openstackgerrit | zhongjun proposed openstack/manila master: TEST https://review.openstack.org/469407 | 07:31 |
| *** markstur has quit IRC | 07:31 | |
| *** gaurangt_ has joined #openstack-manila | 07:42 | |
| *** gaurangt has quit IRC | 07:43 | |
| openstackgerrit | zhongjun proposed openstack/manila master: [WIP] Change ensure share to make startup faster https://review.openstack.org/457545 | 07:48 |
| *** gaurangt has joined #openstack-manila | 07:52 | |
| *** gaurangt_ has quit IRC | 07:53 | |
| *** gaurangt_ has joined #openstack-manila | 07:59 | |
| *** gaurangt has quit IRC | 08:01 | |
| openstackgerrit | zhongjun proposed openstack/manila master: [WIP] Change ensure share to make startup faster https://review.openstack.org/457545 | 08:03 |
| *** kaisers has joined #openstack-manila | 08:12 | |
| *** gaurangt has joined #openstack-manila | 08:14 | |
| *** gaurangt_ has quit IRC | 08:17 | |
| *** kaisers has quit IRC | 08:17 | |
| openstackgerrit | Jan Provaznik proposed openstack/manila master: User Messages https://review.openstack.org/313549 | 08:17 |
| openstackgerrit | Jan Provaznik proposed openstack/manila master: Provide filter name in user messages https://review.openstack.org/443102 | 08:17 |
| openstackgerrit | Jan Provaznik proposed openstack/manila master: Extend usage of user messages https://review.openstack.org/443101 | 08:17 |
| *** gaurangt_ has joined #openstack-manila | 08:29 | |
| *** gaurangt has quit IRC | 08:30 | |
| *** a-pugachev has joined #openstack-manila | 08:52 | |
| *** markstur has joined #openstack-manila | 09:03 | |
| *** gaurangt has joined #openstack-manila | 09:04 | |
| *** gaurangt_ has quit IRC | 09:05 | |
| *** markstur has quit IRC | 09:08 | |
| *** gaurangt has quit IRC | 09:09 | |
| *** gaurangt has joined #openstack-manila | 09:11 | |
| *** kaisers has joined #openstack-manila | 09:14 | |
| *** gaurangt has quit IRC | 09:20 | |
| *** gaurangt has joined #openstack-manila | 09:21 | |
| zhongjun | tbarron: Hi, You want to use ensure_share(s) to force your backend to re-create access rules for shares. It sounds like it is another part when we discuss about refactor ensure share: "weather we can decide which resources(snapshot, access, group) we need to update". If we allow driver to ask ensure_share to force your backend to re-create access rules | 09:33 |
| zhongjun | for shares. How about other resources? | 09:33 |
| *** openstackstatus has quit IRC | 09:36 | |
| *** openstackstatus has joined #openstack-manila | 09:38 | |
| *** ChanServ sets mode: +v openstackstatus | 09:38 | |
| -openstackstatus- NOTICE: There is a known issue with setuptools 36.0.0 and errors about the "six" package. For current details see https://github.com/pypa/setuptools/issues/1042 and monitor #openstack-infra | 09:44 | |
| *** kaisers has quit IRC | 09:49 | |
| zhongjun | tbarron: If we need to recreate share access in ensure share. We could just get the flag( {'access_list_epoch: <new value>} ) return by driver function( get_backend_info) and refresh access if the flag is true(means need to update share_instance['access_rules_status'] != constants.STATUS_ACTIVE or flag=True ). If we have the access_update_flag or | 09:49 |
| zhongjun | instance_update_flag, we could just update access and do not update share instance. It could not need to add a new enum value for share_instance['access_rules_status'], say 'NEEDS_REFRESH'. | 09:49 |
| *** gaurangt_ has joined #openstack-manila | 09:52 | |
| *** gaurangt has quit IRC | 09:52 | |
| tbarron | zhongjun: yes, we could have a specific flag for refreshing access lists (and I guess specific flags for updating snapshots, groups, etc. but I don't need those atm). | 10:30 |
| tbarron | zhongjun: instead of returning an access_rules_status that would trigger the refresh | 10:31 |
| tbarron | zhongjun: thank you for thinking about the problem | 10:31 |
| openstackgerrit | zhongjun proposed openstack/manila master: [WIP] Change ensure share to make startup faster https://review.openstack.org/457545 | 10:39 |
| zhongjun | tbarron: It is my pleasure. | 10:40 |
| *** kaisers has joined #openstack-manila | 10:46 | |
| *** ganso has joined #openstack-manila | 10:46 | |
| *** markstur has joined #openstack-manila | 11:04 | |
| ganso | tbarron: hey Tom! I just read about yesterday's discussion on access rules | 11:05 |
| tbarron | ganso: great, thanks! What are your thoughts? | 11:05 |
| ganso | tbarron: I like your idea, when Goutham and I removed the recovery mode we did so because it was no longer needed in the new design, but we also did not consider the use case you described. I think using ensure_share as you described is the way to go :) | 11:06 |
| ganso | tbarron: but I have a few questions | 11:06 |
| tbarron | ganso: of course | 11:07 |
| ganso | tbarron: when you failover, don't you failover from backendA to backendB, which are both visible and managed by manila? | 11:07 |
| *** kaisers has quit IRC | 11:08 | |
| tbarron | ganso: no, this is a pacemaker controlled failover from one node to another where processes running on multiple nodes are just | 11:08 |
| tbarron | standby until a failover, in which case they become active and fill in the "slot" for the same backend | 11:09 |
| *** markstur has quit IRC | 11:09 | |
| tbarron | ganso: we're already doing this for manila share in general, with whatever backends are configured for it. | 11:09 |
| ganso | tbarron: oh I see, so this backend remains not seen by manila and takes over backendA's role when it fails. ok good | 11:09 |
| ganso | tbarron: I like the fact that we would be setting a flag like "NEEDS_REFRESH", it is something that is controlled on our side | 11:10 |
| tbarron | ganso: the new thing here is that there's a ganesha process that also fails over that is the backend itself. | 11:10 |
| tbarron | (or part of it :)) | 11:10 |
| ganso | tbarron: so, how does manila know something is failing over? how do you intend to check the need to set the "NEEDS_REFRESH" flag? | 11:12 |
| tbarron | ganso: well use cases will probably be different for different backends. The driver would set it. In our case I think we can know that pacemaker started us up because of a failover. | 11:15 |
| tbarron | So we know that our saved access rule state (if we see that we have it) is probably stale. | 11:15 |
| * tbarron notes that we're working in parallel to persist that state on shared storage outside manila so that wouldn't be the case. | 11:16 | |
| ganso | tbarron: hummm, it is a bit unusual workflow to allow a driver to set a DB entry flag... we would need a driver interface "check_failover" that returns a boolean and manager will change DB and then invoke ensure_share for that share. Or something along these lines. | 11:16 |
| tbarron | But even when we solve that perhaps the shared state is no longer accessible for some reason and needs to be rebuilt. | 11:17 |
| ganso | tbarron: ^ in the example I just described the flag is really not needed. | 11:17 |
| tbarron | ganso: the ensure_shares spec already allows driver to cause DB update, so it could be done the same way as export locations and share state in the example in that spec | 11:17 |
| tbarron | ganso: no extra driver interface would be needed since that spec provides a generic mechanism for doing this via get_backend_info() | 11:19 |
| ganso | tbarron: ensure share will be invoked when changes are detected. Unless the failover is detected in this step (this would be the cleanest approach and less code change), you would need another step | 11:19 |
| ganso | tbarron: unless the failover is *not detected, you would need another step | 11:19 |
| ganso | tbarron: if so, again, there is no need for a NEEDS_REFRESH flag | 11:20 |
| ganso | tbarron: this looks to be much simpler than I initially thought | 11:20 |
| tbarron | ganso: agree, that's why my suggestion was (for one way to do this) that when the driver comes up if it decides a refresh is needed, it would add a key-value (doesn't really matter much what it is but I was calling it 'access-sync-epoch' or something like that) that causes the hash to change. | 11:21 |
| tbarron | ganso: But I think there are multiple ways we could do this. I was mostly checking that we agreed that ensure_shares() should be able to refresh access-rule state as well as bare share state. | 11:22 |
| tbarron | ganso: it seems like we all agree that this should be something that we make possible as we implement ensure_share() though bswartz has made the valid point | 11:23 |
| tbarron | that even if we make it possible we should be clear that it's an expensive operation that scales badly so that drivers should avoid using it unless absolutely necessary. | 11:24 |
| ganso | tbarron: yes, I agree. we should not always refresh the access rules in ensure_share (given ensure_share is being invoked) | 11:26 |
| tbarron | ganso: thanks for thinking about this! | 11:39 |
| ganso | tbarron: np! I am glad to be helping :) | 11:44 |
| *** MVenesio has joined #openstack-manila | 11:56 | |
| *** kaisers has joined #openstack-manila | 12:01 | |
| *** kaisers has quit IRC | 12:23 | |
| *** dsariel has quit IRC | 12:29 | |
| *** rraja has joined #openstack-manila | 12:46 | |
| *** markstur has joined #openstack-manila | 12:55 | |
| *** markstur has quit IRC | 13:00 | |
| *** gouthamr has joined #openstack-manila | 13:01 | |
| *** dsariel has joined #openstack-manila | 13:19 | |
| *** kaisers has joined #openstack-manila | 13:20 | |
| *** kaisers has quit IRC | 13:25 | |
| bswartz | mkoderer: question about neutron port binding plugin | 13:36 |
| bswartz | mkoderer: it seems to work great when we use tenant_network_type = vlan, but not when we use vxlan | 13:36 |
| *** dsariel has quit IRC | 13:37 | |
| *** eharney has joined #openstack-manila | 13:43 | |
| *** eharney has quit IRC | 14:00 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack/manila master: Add quotas per share type https://review.openstack.org/452158 | 14:05 |
| openstackgerrit | Tom Barron proposed openstack/manila master: Fix pep8 M325 error with python 3.5 https://review.openstack.org/469523 | 14:11 |
| *** eharney has joined #openstack-manila | 14:15 | |
| *** dsariel has joined #openstack-manila | 14:18 | |
| *** kaisers has joined #openstack-manila | 14:21 | |
| *** cknight has joined #openstack-manila | 14:22 | |
| *** xyang_ has joined #openstack-manila | 14:24 | |
| *** jprovazn has quit IRC | 14:29 | |
| -openstackstatus- NOTICE: python-setuptools 36.0.1 has been released and now making its way into jobs. Feel free to 'recheck' your failures. If you have any problems, please join #openstack-infra | 14:33 | |
| *** openstackgerrit has quit IRC | 14:34 | |
| gouthamr | bswartz: https://i.redd.it/tfvrt0s12z0z.png | 14:43 |
| *** openstackgerrit has joined #openstack-manila | 14:43 | |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Change to share access list API https://review.openstack.org/468287 | 14:43 |
| *** kaisers has quit IRC | 14:44 | |
| *** hongbin has joined #openstack-manila | 14:47 | |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Change to share access list API https://review.openstack.org/468287 | 14:50 |
| ganso | gouthamr: lol | 14:52 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-manila master: Drop -dev in preparation for Pike M2 https://review.openstack.org/469896 | 14:54 |
| *** markstur has joined #openstack-manila | 14:59 | |
| *** rraja has quit IRC | 15:01 | |
| *** eharney has quit IRC | 15:05 | |
| *** eharney has joined #openstack-manila | 15:18 | |
| *** toabctl has quit IRC | 15:22 | |
| *** toabctl has joined #openstack-manila | 15:23 | |
| *** openstackgerrit has quit IRC | 15:33 | |
| tbarron | bswartz: there is no longer a separate unit file for manila api so the instead of 'systemctl restart openstack-manila-api' (my target isn't just right) one does 'kill -SIGINT <pid>' | 15:38 |
| tbarron | and instead of 'journalctl -f --unit=<unitfile-target>' I think one can do 'journalctl SYSLOG_IDENTIFIER=manila-api' (except that for some reason that isn't working in RDO currently). | 15:41 |
| *** kaisers has joined #openstack-manila | 15:42 | |
| tbarron | IMO it was nice to be able to use the same systemctl and journalctl commands for api and non-api services, but | 15:42 |
| tbarron | I'm not arguing that "nice" outweighs the other considerations like lack of eventlet dependency, port consolidation, etc. | 15:43 |
| tbarron | I just wouldn't be surprised if there's some backlash from cloud operators ... | 15:44 |
| *** MVenesio has quit IRC | 15:46 | |
| *** cknight has quit IRC | 15:47 | |
| bswartz | tbarron: the exact systemd configuration will be entirely up to the distros I think | 15:57 |
| gouthamr | tbarron: it's still whatever RH/OOO decides to deploy the api service like, isn't it? | 15:57 |
| gouthamr | tbarron: would they be giving a choice to deployers too? | 15:57 |
| gouthamr | s/they/you | 15:57 |
| tbarron | gouthamr: TBD | 15:57 |
| tbarron | bswartz: agree | 15:57 |
| bswartz | if Redhat makes it that way then it's RedHat's fault if the customers don't like it -- upstream doesn't control how distros configure services | 15:57 |
| bswartz | the way I see it, we're providing one more tool | 15:58 |
| gouthamr | tbarron: i like persistent uncompressed logs.. but then i don't run production clouds at scale.. | 15:58 |
| tbarron | My main question was just whether we have the same set of issues with the alternate web server from zhongjun's patch. | 15:58 |
| bswartz | gouthamr: it's not hard to make systemd produce those | 15:58 |
| gouthamr | bswartz: yes, i'd hope OOO/Packstack would configure all my services that way | 15:59 |
| bswartz | tbarron: I don't really know anything about uwsgi | 15:59 |
| bswartz | gouthamr: packstack already does | 15:59 |
| gouthamr | continue to. | 15:59 |
| bswartz | I don't see why it would change | 15:59 |
| tbarron | bswartz: I didn't mean to come off as saying that this is an upstream problem, was just wondering about uwsgi and raising awareness a bit within devstack | 15:59 |
| bswartz | I leave these decisions to smarter people than me | 16:00 |
| tbarron | as a developer with devstack I was thrown a bit b/c the systemd by default stuff arrived more or less at the same time as apache-by-default | 16:00 |
| bswartz | when I grew up Apache was your only choice | 16:00 |
| tbarron | so as goutham knows I was going "where are my api logs and how do I restart manila-api to set DEBUG, etc." | 16:01 |
| *** winston-d_ has joined #openstack-manila | 16:01 | |
| tbarron | screen went away and systemctl/journalctl didn't work | 16:01 |
| tbarron | for api | 16:01 |
| *** catintheroof has joined #openstack-manila | 16:02 | |
| gouthamr | we could have gone the route of making sure the logs were still emitted to LOG_DIR | 16:03 |
| gouthamr | :) | 16:03 |
| gouthamr | instead of apache's LOG_DIR | 16:03 |
| tbarron | RDO is putting the logs still, uncompressed, under /var/log/{manila,cinder,...}/*api.log | 16:03 |
| bswartz | :-D | 16:03 |
| tbarron | You don't have to use journalctl to look at them. | 16:03 |
| tbarron | But it's nice to use journalctl with wild card on all the manila unit files and that now doesn't include the api log. | 16:04 |
| gouthamr | sure, but i might have something else parse uncompressed log files... | 16:05 |
| gouthamr | s/but/and (agreeing) | 16:05 |
| *** kaisers has quit IRC | 16:05 | |
| tbarron | gouthamr: right, I agree that one should be able to see the logs w/o journalctl too. | 16:05 |
| tbarron | It was also nice to be able to say 'systemctl restart openstack-manila*' | 16:06 |
| tbarron | You can still say it, it just won't restart the api service. | 16:06 |
| *** openstackgerrit has joined #openstack-manila | 16:07 | |
| openstackgerrit | Merged openstack/puppet-manila master: Drop -dev in preparation for Pike M2 https://review.openstack.org/469896 | 16:07 |
| gouthamr | so if multiple services are configured as apache vhosts, should I be reloading apache to grab changes to configuration | 16:08 |
| bswartz | gouthamr: yes | 16:08 |
| bswartz | in a dev environment bouncing apache is harmless | 16:09 |
| bswartz | I think tbarron's point is that simply bouncing apache to pick up a manila-api config change might be too disruptive | 16:09 |
| gouthamr | yes | 16:10 |
| tbarron | so we 'kill -SIGINT <pid>' for the apache pid running manila-api | 16:10 |
| bswartz | I mean in a production environment | 16:11 |
| tbarron | bswartz: yeah, I'm just saying what we will recommend in production env to avoid bouncing all the api services and horizon just to restart manila-api | 16:11 |
| tbarron | in non-production env who cares | 16:12 |
| gouthamr | (systemctl reload httpd)'s behavior is a graceful restart; so not disruptive, but potential delay/painpoint to other traffic | 16:12 |
| gouthamr | okay, don't think that's right, according to https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/ch-Web_Servers.html | 16:14 |
| tbarron | gouthamr: it says it's a graceful restart that causes reload of apache conf file. I'll have to test whether it causes reload of manila.conf, cinder.conf, etc. | 16:16 |
| tbarron | gouthamr: it does, so the situation isn't so bad as I thought: one can use reload and it's not so alarming, though every api service "pauses" from end user standpoint. | 16:20 |
| *** dsariel has quit IRC | 16:21 | |
| openstackgerrit | Merged openstack/manila master: Fix pep8 M325 error with python 3.5 https://review.openstack.org/469523 | 16:25 |
| *** catintheroof has quit IRC | 16:27 | |
| *** cknight has joined #openstack-manila | 16:53 | |
| *** kaisers has joined #openstack-manila | 17:02 | |
| *** a-pugachev has quit IRC | 17:16 | |
| gouthamr | tbarron: yes.. that's what i'd assume, and i think that's bad to an unsuspecting client | 17:19 |
| tbarron | gouthamr: ack | 17:20 |
| *** kaisers has quit IRC | 17:26 | |
| *** kaisers has joined #openstack-manila | 17:28 | |
| *** kaisers has quit IRC | 17:32 | |
| *** MVenesio has joined #openstack-manila | 17:35 | |
| *** kaisers has joined #openstack-manila | 17:45 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/manila master: Updated from global requirements https://review.openstack.org/469570 | 17:53 |
| *** xyang_ has quit IRC | 18:01 | |
| *** xyang_ has joined #openstack-manila | 18:03 | |
| *** xyang_ has quit IRC | 18:04 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack/manila master: Add quotas per share type https://review.openstack.org/452158 | 18:21 |
| openstackgerrit | Merged openstack/manila master: Use get_notification_transport for notifications https://review.openstack.org/469630 | 18:29 |
| *** xyang1 has joined #openstack-manila | 19:08 | |
| *** xyang_ has joined #openstack-manila | 19:21 | |
| *** MVenesio has quit IRC | 19:30 | |
| *** eharney has quit IRC | 20:03 | |
| *** winston-d_ has quit IRC | 20:39 | |
| *** xyang_ has quit IRC | 20:47 | |
| *** xyang_ has joined #openstack-manila | 20:47 | |
| *** cknight has quit IRC | 20:49 | |
| *** kaisers1 has quit IRC | 21:12 | |
| openstackgerrit | Tom Barron proposed openstack/manila master: Clean releasenotes and install-guide build dir https://review.openstack.org/470019 | 21:20 |
| *** kaisers1 has joined #openstack-manila | 21:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/manila master: Updated from global requirements https://review.openstack.org/469570 | 21:32 |
| *** xyang_ has quit IRC | 21:57 | |
| *** xyang_ has joined #openstack-manila | 21:59 | |
| *** markstur has quit IRC | 22:00 | |
| *** markstur has joined #openstack-manila | 22:05 | |
| *** gouthamr has quit IRC | 22:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/manila master: Updated from global requirements https://review.openstack.org/469570 | 22:12 |
| *** xyang_ has quit IRC | 22:20 | |
| *** ganso has quit IRC | 22:26 | |
| *** gouthamr has joined #openstack-manila | 22:58 | |
| *** markstur has quit IRC | 23:35 | |
| *** xyang1 has quit IRC | 23:44 | |
| *** markstur has joined #openstack-manila | 23:46 | |
| *** hongbin has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
