Privacy-Preserving Multimedia Mobile Cloud Computing Using Protective Perturbation

A typical multimedia mobile cloud computing system involves a mobile client and a cloud server. The mobile client offloads input data to the cloud server to request computation-intensive services from the cloud server, which makes the mobile device achieve significantly lower resources (e.g., power) consumption and higher performance. Also, it enables the service providers to deploy the proprietary image recognition models in their trusted domains for effective maintenance and intellectual property protection.

However, as a trade-off, the empirical deployment of such a mobile cloud computing system faces the following two challenges:

• •

  Privacy/Security challenge. Since the user data, which are often of a privacy-sensitive nature, must be offloaded to the cloud server, privacy concerns are often raised due to both the user’s natural psychological needs and the legal regulations (Pri, 2020). Also, the untrusted or unsecured cloud service provider and network may enable adversaries to obtain and abuse the private user data to issue various cybersecurity attacks (Str, 2020).

• •

  Bandwidth challenge. The data offloading from the mobile client to the cloud server may consume a large amount of bandwidth in the communication network, especially when rich multimedia content is involved (e.g., high-definition images or videos).

We only consider visual privacy as the threat model in this work, and do not consider security threats like crossmatching attacks or other attacks that may compromise the visual privacy solutions (e.g., protective perturbations). Many research efforts have focused on privacy-preserving image delivery to address the visual privacy challenge, where the goal is to prevent the user’s private image from being exposed to others. For example, several research works propose to inject perturbations to blur and protect the privacy-sensitive information in the images (Ren et al., 2018; Sun et al., 2018; Wu et al., 2021; Zhu et al., 2020; Ye et al., 2022). Such perturbation generator can be deployed on the mobile client to blur the sensitive images before it is exposed to the untrusted network or service provider.

Without loss of generality, we adopt the state-of-the-art protective perturbation generator (Ye et al., 2022) in our mobile cloud image recognition system. This perturbation generator utilizes U-Net $U(\cdot)$ as the core generative model, which is trained to inject perturbations to the original images, so that they are entirely blurred to human vision for privacy protection but remain as effective inputs to the image recognition model on the cloud server to achieve highly accurate recognition results. The goal is achieved by minimizing the loss function $\omega_{1}\cdot loss(y_{i_{target}},y_{i})-\omega_{2}\cdot loss(y_{i_{aux}},y_{i})+\omega_{3}\cdot SSIM(x_{i},x_{i}^{\prime})$, where $\omega_{j}\ (j\in\{1,2,3\})$ are tunable weights of each term, $loss(\cdot)$ is the Cross-Entropy Loss, $y_{i}$ is the true label of the input image $x_{i}$, $y_{i_{target}}$ and $y_{i_{aux}}$ are predicted labels of the perturbed image $x_{i}^{\prime}$. Note that this work utilizes two image recognition models during the training process, a target model (outputs $y_{i_{target}}$) and an auxiliary model (outputs $y_{i_{aux}}$), where the target model is the model on the cloud server, and the auxiliary model is a different image recognition model to help generate the perturbation, chosen by experiments.

We conduct two case studies to reveal the limitations of the existing protective perturbation and image encoding approaches, which cannot effectively address the aforementioned privacy and bandwidth challenges in our target end-to-end mobile cloud image recognition system.

Figure 2 shows the overall architecture of PMC². It introduces a trusted secure edge server between the mobile client and the traditional cloud server. The secure edge server accomplishes privacy-preserving perturbation generation in a confidential computing container protected by the trusted execution environment (TEE) technology. In addition, a neural encoder and decoder are deployed on the secure edge server and the cloud server, respectively, to attain the bandwidth-saving goal.

The end-to-end PMC² system works as follows to achieve privacy-preserving mobile-cloud image recognition. First, the mobile client compresses the privacy-sensitive images using JPEG and offloads them to the trusted secure edge server via an in-domain HTTPS link. The secure edge server is deployed inside the same trusted domain as the mobile client. Second, on the secure edge server, a perturbation generator runs inside the TEE-protected confidential container to securely inject protective perturbations to the received user images in real time; Then, the perturbed images are compressed by a perturbation-aware neural encoder and transferred to the cloud server for image recognition via the cross-domain HTTPS link. The cloud server is deployed in a remote and untrusted domain. Third, the cloud server decodes the compressed images using a neural decoder, which reconstructs the perturbed images and feeds them into the machine learning-based image recognition model for the computation-intensive classifications. Finally, the image recognition results are returned to the mobile client via the secure edge server.

We employ the state-of-the-art confidential computing techniques to generate the protective perturbations on the secure edge server to address the privacy/security challenge without incurring power and latency overhead on the mobile client. Confidential computing has been widely adopted in the modern cloud platforms by leveraging trusted computing environment (TEE) technology  (AWS, 2023; Goo, 2023; Azu, 2023; IBM, 2023). Such technology provides secure enclaves in the system, which guarantees that the computation conducted in each enclave is encrypted and isolated from the rest of the system. Generally there are two types of TEEs, namely process-based TEEs and virtual machine (VM)-based TEEs. Process-based TEEs such as Intel SGX (sgx, 2021) can achieve minimal trusted computing base (TCB) since they only deploy the processes that must be protected into enclaves. On the other hand, such techniques require partitioning each application into secure and non-secure domains and thus introduce significant offline overhead (i.e., application partitioning) and runtime overhead (i.e., context switches between the secure and non-secure domains). Due to these limitations on process-based TEEs, there has recently been increasing demand for VM-based TEEs, such as Intel TDX (Int, 2023) and AMD SEV (AMD, 2023), which deploy the entire VM into an enclave. Although VM-based TEEs result in larger TCB than the process-based TEEs, they are more suitable for cloud-native computing and can be easily integrated into Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) platforms.

To protect the confidentiality of the perturbation generation in PMC², we deploy the perturbation generator into a TEE on the secure edge server. Note that our proposed framework is generally compatible with any TEE techniques but, in this work, we leverage confidential containers (CoCo) (CoC, 2023) with AMD SEV (AMD, 2023) (i.e., a VM-based TEE) for the following reasons. First, CoCo supports running containerized applications and can be managed by Kubernetes (K8s) (K8s, 2023) platforms, which helps us achieve a cloud-native computing environment. Second, CoCo supports multiple VM-based TEE techniques, such as AMD SEV and Intel TDX (Int, 2023), which provides flexibility for the users to choose different hardware platforms. Third, applications can be directly deployed into CoCo without any modifications, which significantly increases the diversity of supported perturbation generation models that can be applied to the PMC² framework and does not introduce additional offline overhead to modify the models.

Kindly notice that the protective perturbation generators run with CPU only without leveraging GPU or other acceleration hardware since accelerator TEEs have not become widely available in the server/cloud environment yet. We will leverage secure accelerators in the future upon their availability.

As shown in Figure 2, there are two communication links in PMC² that consume network bandwidth: (a) in-domain, the compressed user image is transmitted from the mobile client to the secure edge server, and (b) cross-domain, the encoded perturbed image is transmitted from the secure edge server to the cloud server. Let us denote the original image as $Img_{orig}$, which is encoded and stored in PNG by default. The JPEG-compressed version, $JPEG(Img_{orig})$, is delivered via the in-domain link. The perturbed image $Img_{pert}$ is equivalent to $PertGen(JPEG(Img_{orig}))$, where $PertGen(\cdot)$ is the perturbation generator, and it will be encoded by the encoder function $Enc(\cdot)$. Then, $Enc(Img_{pert})$ is the image transmitted via the cross-domain link.

Based on our experiments, the target model accuracy for
$PertGen(JPEG(Img_{orig}))$ and $PertGen(Img_{orig})$ have very small differences (¡ 2%) with huge bandwidth savings (about 40%); therefore, the bandwidth-saving encoding for the the in-domain link can be easily solved by adopting JPEG. To address the bandwidth challenge for the cross-domain link, i.e., $Enc(Img_{pert})$, we consider PNG and JPEG as the baseline $Enc(\cdot)$ approaches, which are subject to the compression inefficiency issue discussed in Section 2.3.2. PNG achieves lossless compression but with low compression ratio and high bandwidth consumption. For JPEG, we observe that the default $subsampling$ parameters would break the functionality of the target model. When $subsampling$ is disabled, to attain the same level of the target model accuracy, the compression ratio for perturbed images is significantly lower than that of regular images, as shown in Section 2.3.2. These observations motivate us to develop a new neural compressor tailored to the perturbed images to reduce the bandwidth consumption on the cross-domain link.

Table 3 presents the image recognition accuracy and the average size of the compressed image when deploying different image encoding methods between the secure edge server and the cloud server. The average size of $JPEG(Img_{orig})$ (i.e., the images sent from the mobile client to the secure edge server) and the average size of $Enc(Img_{pert})$ (i.e., the images sent from the secure edge server to the cloud server) are represented by $S_{1}$ and $S_{2}$, respectively. The standard deviation values of $S_{1}$ and $S_{2}$ are also provided after the $\pm$ symbols to describe the distribution of file sizes. All the standard deviation values are relatively small, indicating all the file sizes are generally close to the average values. The PNG-encoded $Img_{pert}$ achieves recognition accuracies between 90.00% to 90.48%, and the image sizes range from 2544.70 to 3170.88 Bytes. The accuracy values decrease by 0.90% to 1.88% compared to the numbers reported in Table 2. Since the results in Table 2 come from the cases where uncompressed user images are used to generate perturbed images, $JPEG(Img_{orig})$ (i.e., the reduction of $S_{1}$) is the only factor that impacts the target model accuracy. Considering the raw size of a 32x32 RGB image is just 3096 Bytes (32x32x3), it is apparent that the protected images are not effectively compressed in the PNG format. We then take a step forward by replacing PNG with JPEG compression, which achieves significant data size reduction, ranging from 19.85% (MobileNet_v2) to 32.37% (GoogLeNet), while maintaining similar recognition accuracies (i.e., above 90%).

The neural compressor in the proposed PMC² system shows superior accuracy and bandwidth savings. More specifically, the neural-compressed images still keep high target model accuracy, which is only reduced by 0.80% (GoogLeNet) to 1.88% (VGG16_bn) compared to the PNG encoding. On the other hand, a protected image can be compressed to a bit-string ranging from 8 to 8.35 Bytes, which is achieved by a neural compression model sized at only 12 MB. The data size is reduced by more than 99% compared to both PNG and JPEG encoding methods. In short, the proposed neural compression outperforms JPEG compression in terms of data size and thus bandwidth consumption between the secure edge server and the cloud server, without affecting the functionality of the image recognition model.

Note that the quality factors used for JPEG in both links, if applicable, are chosen experimentally to achieve similar target model accuracy (i.e., 90%) for a fair comparison. It is also worth noting that the average size of $Img_{orig}$ is 2662.56 Bytes and, after applying JPEG compression with proper JPEG quality factors, the size (i.e., $S_{1}$) decreases by 40% on average, with almost no negative impact on the target model accuracy.

We further evaluate the end-to-end latency of the proposed PMC² system using the 8 target image recognition models and image encoding methods. For comparison, we also conduct the same timing evaluation on the baseline protective perturbation system to uncover the gains of the proposed mechanism. Figure 4 shows the average end-to-end latency results with standard deviation values and timing breakdown analysis in 4 test cases for each target model. The first 3 test cases, namely PNG, JPG, and NC, represent using PNG, JPEG, and neural compression for the secure edge server-cloud server link in the PMC² system; the 4th test case, namely protective perturbation (PP), represents the baseline protective perturbation system with PNG encoding. The time breakdown results include nine components, which are listed at the bottom of Figure 4. The secure edge server is named “AMD”, and the cloud server is named “GPU”. Particularly, there are no “JPEG Compression on Mobile”and “Network (Mobile-AMD)” components in the baseline case (i.e., PP).

The end-to-end latency of the PMC² system ranges from 6.766 to 8.234 $ms$, which is significantly less than the baseline protective perturbation system ranging from 12.669 $ms$ to 13.893 $ms$. Among all three cases for PMC², the neural compression approach consistently outperforms the standard PNG by 1.40% (Inception_v3) to 6.17% (GoogLeNet), and it outperforms JPEG by 0.90% (ResNet18) to 6.42% (GoogLeNet). In general, PMC² takes 42.69% (Inception_v3 w/ neural compressor) to 48.67% (GoogLeNet w/ neural compressor) less time than the baseline protective perturbation system for the same target model.

The “JPEG Compression on Mobile” takes about 0.37 $ms$ on average, while the “Network (Mobile-AMD)” takes about 0.54 $ms$ on average. Based on its design, the protective perturbation generator has an almost constant running time regardless of the target image recognition model. On average, it takes 2.615 $ms$ inside an SEV-enabled CoCo container or 8.135 $ms$ on a smartphone to generate a protected image. Overall, it is the most time-consuming component in all the cases, which can take up to 39.27% of the processing time spent in PMC², and 63.60% of that in the baseline protective perturbation system.

The average PNG compression and decompression times are 0.125 $ms$ and 0.142 $ms$ in PMC², 0.272 $ms$ and 0.164 $ms$ in the baseline system, while those for JPEG are 0.150 $ms$ and 0.158 $ms$ in PMC². For both PNG and JPEG, compression and decompression take less than 4.45% of the total running time in PMC², and at most 3.49% in the baseline system. Other operations on the secure edge server take about 0.4 $ms$ for both encoding methods, and other operations on the cloud server take about 1.8 to 3.1 $ms$. On the other hand, the compression and decompression times are 0.558 $ms$ and 1.325 $ms$ on average for the neural compressor, which take 7.99% to 18.96% time in the entire system. The neural compressor needs 0.362 $ms$ for other operations on the secure edge server and 0.099 to 1.322 $ms$ on the cloud server. It turns out that PNG and JPEG encoding/decoding take approximately equal time, with PNG slightly faster. The neural compressor needs about 4x time for compression and 9x time for decompression but less time for other operations to prepare the data for compression/decompression. The network latency and the image recognition model take stable and short time in all cases.

We use the Monsoon power monitor (Mon, 2023) to collect the power traces of the mobile device. The power monitor is connected to a Ubuntu workstation and controlled by Python scripts. The smartphone keeps the same network connection and screen brightness (e.g., the lowest) during the testing for a fair comparison.

Figure 5 shows the power consumption comparison between the baseline protective perturbation system and the PMC² system using 8 different target models. PNG is employed for the baseline system, and the neural compressor is adopted for the PMC² system. The results for all the target models show a similar pattern that indicates significant power savings on the resource-constrained mobile device achieved by the proposed PMC² system. For example, we can take a closer look at the two power tracing results of VGG13_bn and Densenet121 as, in the timing results of PMC², VGG13_bn is the fastest target model among all 8 models, and DenseNet121 is the slowest. In the VGG13_bn case, the target image recognition model used in both systems is VGG13_bn. In the baseline system, most power traces fall into the 1.5 to 8.0 Watts range (blue lines), averaging 4.961 Watts (red lines). As a comparison, the PMC² system consumes 0.751 Watts on average (green lines). Offloading the protective perturbation generation to the secure edge server makes the system much faster, as the test only lasts 33.956 seconds, almost half of the 63.345 seconds for the baseline system. Suppose we calculate the energy consumption by multiplying the average power and processing time; it turns out that the PMC² costs only 8.11% energy of the baseline system. Similarly, when DenseNet121 is deployed as the target model, the baseline system consumes 66.065 seconds and 4.556 Watts average power to process all the 5000 images from the test dataset. Meanwhile, PMC² only needs 34.716 seconds and 0.748 Watts average power to finish the same task, which yields a 91.37% energy reduction.