OpenStack Swift源码初探--proxy下的server.py

OStack程序员社区-中国程序员成长平台 › 门户 › 编程› 移动开发›Swift 教程

原作者: [db:作者] 来自: [db:来源] 收藏邀请

OpenStack Swift作为开源的云存储工具，被越来越多的公司使用。为了记录和巩固学习swift的开源源码，所以进行一系列的源码开源学习笔记，供初学者快速学习和理解swift的内部功能。

proxy下面的server.py模块是所有对account，container，object等对象进行管理操作的在swift的proxy端的总入口。在swift系统在接收到url请求后，先是经过middleware处理链分别验证处理后，再进入到proxy下面的server模块，进入 _call_方法调用后，把对应的请求分发给不同的controller处理，controller再调用各自的nodeStroage服务程序进行处理，返回各自的resp结果到server模块，最后通过mimmdleware处理链再反方向返回最终的请求处理结果

1.首先server.py的整个结构如下图：包括4部分：一堆import引用，一个required_filters的字典，一个名为“Application(object)”的class ，一个app_factory(global_conf, **local_conf) 方法

2.主要介绍“Application(object)”的class，其中包含了所有主要的功能方法

2.1 _init_ 方法，Application类的初始化方法，主要就是初始化一些对象，包括：conf配置文件参数的初始化，log日志初始化，memcache对象初始化，account_ring,container_ring, object_ring对象初始化等

2.2 check_config(self) 方法，主要是检查配置文件proxy-server.conf中配置的“read_affinity” 和“sorting_method”属性值是否正确，该方法在 app_factory(global_conf, **local_conf):方法时调用

2.3 get_controller(self, path)方法，主要是根据传入的 urlPath解析并返回对应的control类和一个字典对象，其中字典对象的值根据传入url格式的不同返回不同的值

[python] view plain copy

def get_controller(self, path):
"""
Get the controller to handle a request.
:param path: path from request
:returns: tuple of (controller class, path dictionary)
:raises: ValueError (thrown by split_path) if given invalid path
"""
if path == '/info': #url是/info 则返回InfoController和包括version,expose_info,disallowed_sections,admin_key的字典
d = dict(version=None,
expose_info=self.expose_info,
disallowed_sections=self.disallowed_sections,
admin_key=self.admin_key)
return InfoController, d
version, account, container, obj = split_path(path, 1, 4, True) #以/拆分url为数列，并取对应的1到4位的数据返回给对应的变量
d = dict(version=version,
account_name=account,
container_name=container,
object_name=obj)
if obj and container and account: #根据解析出的account值，congtainer和object值得有无，确定适用的Controller是那种
return ObjectController, d
elif container and account:
return ContainerController, d
elif account and not container and not obj:
return AccountController, d
return None, d

2.4 __call__(self, env, start_response)方法，是server模块的实际对account、container、object等对象调用处理的功能入口。

[python] view plain copy

def __call__(self, env, start_response):
"""
WSGI entry point.
Wraps env in swob.Request object and passes it down.
:param env: WSGI environment dictionary
:param start_response: WSGI callable
"""
try:
if self.memcache is None: #首先判断是否memcache值存在，不存在再去获取一次
self.memcache = cache_from_env(env)
req = self.update_request(Request(env)) #判断header中是否有x-storage-token和x-auth-token
return self.handle_request(req)(env, start_response) #调用handle_request方法并返回处理的结果resp对象
except UnicodeError:
err = HTTPPreconditionFailed(
request=req, body='Invalid UTF8 or contains NULL')
return err(env, start_response)
except (Exception, Timeout):
start_response('500 Server Error',
[('Content-Type', 'text/plain')])
return ['Internal server error.\n']

2.5 update_request(self, req)方法，根据请求中header里面的x-storage-token有而x-auth-token没有的情况，把x-storage-token的值赋予x-auth-token

2.6 handle_request(self, req)方法，server模块实际处理request请求的方法，熟悉servlet的同学可以把它理解成servlet的作用

[python] view plain copy

def handle_request(self, req):
"""
Entry point for proxy server.
Should return a WSGI-style callable (such as swob.Response).
:param req: swob.Request object
"""
try:
self.logger.set_statsd_prefix('proxy-server') #在日志的开头加上‘proxy-server’,方便跟踪分析
if req.content_length and req.content_length < 0: #检查header里面中的Content-Length是否有值，无值返回错误请求，并日志记录
self.logger.increment('errors')
return HTTPBadRequest(request=req,
body='Invalid Content-Length')
try:
if not check_utf8(req.path_info): #检查Pathde的编码是否不满足utf8，不满足返回错误请求，并日志记录
self.logger.increment('errors')
return HTTPPreconditionFailed(
request=req, body='Invalid UTF8 or contains NULL')
except UnicodeError:
self.logger.increment('errors')
return HTTPPreconditionFailed(
request=req, body='Invalid UTF8 or contains NULL')
try:
controller, path_parts = self.get_controller(req.path) #调用get_controller(self,path)方法返回正确的controller类和字典对象
p = req.path_info
if isinstance(p, unicode):
p = p.encode('utf-8') #path编码Unicode转换utf-8
except ValueError: #发生值异常，返回错误请求，并日志记录
self.logger.increment('errors')
return HTTPNotFound(request=req)
if not controller: #为找到对应处理的controller类时，返回错误请求，并日志记录
self.logger.increment('errors')
return HTTPPreconditionFailed(request=req, body='Bad URL')
if self.deny_host_headers and \ #当proxy-server.conf中deny_host_headers有值，且请求的header中的host在deny_host_headers中，则返回错误请求，并日志记录
req.host.split(':')[0] in self.deny_host_headers:
return HTTPForbidden(request=req, body='Invalid host header')
self.logger.set_statsd_prefix('proxy-server.' +
controller.server_type.lower()) #在日志的开头加上‘proxy-server.controller类中的请求类型（eg：HEAD/GET/PUT）’,方便跟踪分析
controller = controller(self, **path_parts) #初始化实际的controller对象（AccountController、ContainerController、ObjectController、InfoController其中之一）
if 'swift.trans_id' not in req.environ: #如果没有trans_id在env中，则重新生成一个，有些类似于http请求中的seesionID的感觉，是一种UUID
# if this wasn't set by an earlier middleware, set it now
trans_id = generate_trans_id(self.trans_id_suffix)
req.environ['swift.trans_id'] = trans_id
self.logger.txn_id = trans_id
req.headers['x-trans-id'] = req.environ['swift.trans_id']
controller.trans_id = req.environ['swift.trans_id']
self.logger.client_ip = get_remote_client(req) #把请求中获取出请求端的IP信息，加入logger对象，方便后续日志查看分析
try:
handler = getattr(controller, req.method) #根据req.method方法获取对应controller对象中的方法（可能是多个，有的有public标签，有的没有）
getattr(handler, 'publicly_accessible') #再根据public标签获取最终的处理方法。（在方法前面可以加 @public 和@delay_denial）
except AttributeError:
allowed_methods = getattr(controller, 'allowed_methods', set())
return HTTPMethodNotAllowed(
request=req, headers={'Allow': ', '.join(allowed_methods)})
if 'swift.authorize' in req.environ: #做鉴权操作
# We call authorize before the handler, always. If authorized,
# we remove the swift.authorize hook so isn't ever called
# again. If not authorized, we return the denial unless the
# controller's method indicates it'd like to gather more
# information and try again later.
resp = req.environ['swift.authorize'](req)
if not resp:
# No resp means authorized, no delayed recheck required.
del req.environ['swift.authorize']
else:
# Response indicates denial, but we might delay the denial
# and recheck later. If not delayed, return the error now.
if not getattr(handler, 'delay_denial', None):
return resp
# Save off original request method (GET, POST, etc.) in case it
# gets mutated during handling. This way logging can display the
# method the client actually sent.
req.environ['swift.orig_req_method'] = req.method
return handler(req) #调用最终的method方法，并返回resp结果
except HTTPException as error_response:
return error_response
except (Exception, Timeout):
self.logger.exception(_('ERROR Unhandled exception in request'))
return HTTPServerError(request=req)