Abstract

Serverless computing has revolutionized online service development and deployment with ease-to-use operations, auto-scaling, fine-grained resource allocation, and pay-as-you-go pricing. However, a gap remains in configuring serverless functions - the actual resource consumption may vary due to function types, dependencies, and input data sizes, thus mismatching the static resource configuration by users. Dynamic resource consumption against static configuration may lead to either poor function execution performance or low utilization. This paper proposes Freyr+, a novel resource manager (RM) that dynamically harvests idle resources from over-provisioned functions to accelerate under-provisioned functions for serverless platforms. Freyr+ monitors each function's resource utilization in real-time and detects the mismatches between user configuration and actual resource consumption. We design deep reinforcement learning (DRL) algorithms with attention-enhanced embedding, incremental learning, and safeguard mechanism for Freyr+ to harvest idle resources safely and accelerate functions efficiently. We have implemented and deployed a Freyr+ prototype in a 13-node Apache OpenWhisk cluster using AWS EC2. Freyr+ is evaluated on both large-scale simulation and real-world testbed. Experimental results show that Freyr+ harvests 38% of function invocations' idle resources and accelerates 39% of invocations using harvested resources. Freyr+ reduces the 99th-percentile function response latency by 26% compared to the baseline RMs.

Department(s)

Computer Science

Comments

National Science Foundation, Grant 2315614

Keywords and Phrases

Attention; incremental learning; reinforcement learning; resource harvesting; serverless computing

International Standard Serial Number (ISSN)

1558-2183; 1045-9219

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Institute of Electrical and Electronics Engineers, All rights reserved.

Publication Date

01 Jan 2024

Download

Full Text Link

Share

 
COinS