Title
\subtitlename

P1. The background of spatio-temporal prediction towards smart city construction.

Fast urbanization leads to explosively increasing transportation, security incident, social events etc.

Many efforts have been paid to address this, i.e., spatio-temporal data mining.

P2. There are multiple tasks demand to be handled, which emerge in a complex spatio-temporal pattern. Related works cannot handle well.

1. There exist common spatio-temporal characteristics among multiple attributes.

2. On the other hand, different attributes own distinct features.

Current research from three lines can not handle this well. Three lines: a. Spatio-temporal prediction model; b. Spatio-temporal multi-task model; c. Multi-task learning model;

P3. We identify three properties that spatio-temporal multi-variate learner should own.

1. Generality: the common characteristics of multiple attributes should be well-addressed.

2. Adaptivity: scalable to fit distinct characteristics of the specific attributes.

3. Efficiency: the method is supposed to be swift and memory-friendly.

P4. The challenges towards the former goals.

It is non-trivial to accomplish the goals. The challenge lie in three aspects:

1. To capture the common characteristic among multiple attributes, it demands high capacity and effective spatio-temporal modeling.

2. The model should fit well on specific attributes, while keeping the common knowledge.

3. The model is supposed to own advancing time and space efficiency, so that it could be deployed generally.

P5. We devise such a model in this paper, introduce our contributions.

Considering the promising precedents in neural language processing and computer vision, it is intuitive to incorporate pretrain-finetune pattern to train a model. However, it suffers from

In this paper, we explore modeling the city data in a unified view.

1. We present a novel transformer framework to capture the common spatio-temporal patterns in multiple attributes.

2. We introduce a pretrain-prompt tune scheme with two types of lightweight prompt tokens to fit specific spatio-temporal attribute, which saves at most 99% parameters with advancing performance.

3. To the best of the authors' knowledge, it is the first model to address multivariate spatio-temporal prediction in parameter-efficient way.

Definition 1. Region. [Junbo Zhang Sig16]

Definition 2. Spatio-temporal multivariate prediction.

P1. Spatio-temporal Transformer

1. Temporal Encoder

1.1 Positional Embedding

1.2 Multi-Head Attention

1.3 Feed-Forward Layer

2. Spatial Encoder

2.1 Positional Embedding

2.2 Spatial Representation Learning (same as 1.2 and 1.3)

3. Header

P2. Pretrain

P3. Prompt Tuning

1. Spatio-temporal Prompt

2. Tiny Spatio-temporal Prompt

3. Prompt Tuning

Overall Result

Transferability Analysis

Ablation Study

Hyper-parameter Analysis