Can We Improve Information Freshness with

Predictions in Mobile Crowd-Learning?

Zhengxiong Yuan† Bin Li‡ Jia Liu†
† Dept. of Computer Science, Iowa State University
‡ Dept. of Electrical and Computer Engineering, University of Rhode Island

Abstract—The rapid growth of mobile devices has spurred arrivals, data trends and other predictable information. These
the development of crowd-learning applications, which rely on developments lead to several fundamental open questions: 1)
users to collect, report and share real-time information. A Can we improve information freshness with predictions in
critical factor of crowd-learning is information freshness, which
can be measured by a metric called age-of-information (AoI). mobile crowd-learning? 2) If the answer to 1) is “yes”, how
Moreover, recent advances in machine learning and abundance to exploit predictions to achieve better AoI performance? 3)
of historical data have enabled crowd-learning service providers What are the bounds and limits of prediction-assisted AoI
to make precise predictions on user arrivals, data trends and performance in mobile crowd-learning?
other predictable information. These developments lead to a
However, analyzing crowd-learning AoI performance with
fundamental question: Can we improve information freshness
with predictions in mobile crowd-learning? In this paper, we predictions faces the following challenges: First, there is a lack
show that the answer is affirmative. Specifically, motivated by of analytical model that takes predictions into consideration
the age-optimal Round-Robin policy, we propose the so-called in mobile crowd-learning in the literature. Second, the inter-
“periodic equal spreading” (PES) policy. Under the PES policy, actions between arrival patterns, real-time information states
we first reveal a counter-intuitive insight that the frequency of
and their predictions are highly complex, where changes in one
prediction should not be too often in terms of AoI improvement.
Further, we analyze the AoI performances of the proposed PES factor would significantly affect the others. Third, as will be
policy and derive upper bounds for the average age under i.i.d. shown later, there exists a long-range coupling among different
and Markovian arrivals, respectively. In order to evaluate the AoI prediction windows, which significantly increases the difficulty
performance gain of the PES policy, we also derive two closed- in analyzing the AoI performance.
form expressions for the average age under uncontrolled i.i.d.
and Markovian arrivals, which could be of independent interest.
As a starting point, in this paper, we focus on a single-PoI
Our results in this paper serve as a first building block towards system with predictable arrivals (up to a window size into the
understanding the role of predictions in mobile crowd-learning. future). In this setting, we address the above challenges and
obtain several fundamental results on understanding the role
I. I NTRODUCTION of predictions in mobile crowd-learning. The main results and
In recent years, the rapid growth of mobile devices (e.g., contributions of this paper are summarized as follows:
smartphones, tablets, wireless sensors, etc.) has spurred the • First, we introduce an analytical model for a single-PoI
development of crowd-learning applications, which rely on crowd-learning system with finite-range predictable arrivals,
users to collect, report and share real-time information for which takes into account the strong coupling between the
a set of points of interest (PoIs). Such applications include, stochastic user arrivals and the AoI of the data. In this
but are not limited to, real-time gasoline price information setting, motivated by the age-optimal Round-Robin policy,
sharing (GasBuddy [1]), real-time traffic states (Google Waze we propose the so-called “periodic equal spreading” (PES)
[2]), WiFi hotspots searching (WiFi Finder [3]), etc. Al- policy, which reshapes the arrivals in such a way that the
though crowd-learning has become increasingly popular, its inter-arrival times are nearly equalized.
future prospect heavily hinges upon a performance metric
• Then, under the PES policy, we first consider the problem of
termed information freshness, which is also known as “age-
choosing an appropriate prediction period, which is referred
of-information” (AoI) in the research community. Ensuring
to as “step size” in this paper. Towards this end, we reveal
crowd-learned information freshness is critical because fresh
a surprising insight that the prediction frequency should not
information retains existing users and attracts new users to
be made too often in terms of AoI improvement.
participate, which in turn improves the information freshness
and creates a positive feedback loop. Meanwhile, recent ad- • Finally, we analyze the AoI performance of the PES policy
vances in machine learning and abundance of historical data and establish upper bounds for the average age under i.i.d.
collected by pervasive mobile devices have enabled crowd- and Markovian arrivals, respectively. In order to evaluate
learning service providers to make precise predictions on user the AoI performance gain of the PES policy, we also derive
two closed-form expressions for the average age under
uncontrolled i.i.d. and Markovian arrivals, which could be
This work is supported in part by NSF ECCS-1818791, CCF-1758736, of independent theoretical interest.
CNS-1758757, CNS-1717108, CNS-1815563; ONR N00014-17-1-2417,
AFRL FA8750-18-1-0107, and Google Faculty Research Award. Collectively, our results in this paper serve as a first building

702
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 Service Provider relies on randomly arriving users to sample and report the
Unshaped Arrivals Reshaped Arrivals
Arrival
state of the PoI.
Reshaping
Policy
PoI We assume that in every time-slot t, the service provider
Â[t] A[t]
w can accurately predict a window of future user arrivals, which
Prediction Window is of w time-slots1 . Although the natural arrivals of the users
Update Report
Record follow some underlying stochastic process, we assume that
the arrival pattern of the users can be reshaped by the service
provider through some reward/incentive mechanism. In other
Fig. 1: A single-PoI predictive crowd-learning system. words, the reward/incentive provided by the service provider
block towards understanding the role of predictions in mobile is sufficiently high so that all users are fully cooperative and
crowd-learning. The remainder of this paper is organized as willing to change their arrival times. We assume that the time-
follows: Section II reviews related work. Section III presents slot duration is sufficiently short so that there is at most one
system model and problem statement. Section IV introduces user arriving in any given time-slot. We use A[t] and Â[t]
the PES policy. Section V studies the AoI performance for to denote the reshaped and unshaped arrival in time-slot t,
i.i.d. and Markovian arrivals, respectively. Section VI illus- respectively. Here, A[t] = 1 (Â[t] = 1) represents that there
trates numerical results and Section VII concludes this paper. is a reshaped (unshaped) user arrival at the PoI in time-slot t;
otherwise, A[t] = 0 (Â[t] = 0) means if there is no reshaped
II. R ELATED W ORK (unshaped) arrival in time-slot t.
As a new performance metric, AoI has recently attracted The service provider maintains a record for the PoI. We use
increasing attention from the information theory, signal pro- ∆[t] to denote the age (freshness) of the recorded information
cessing, and communications communities due to its close in time-slot t, which is defined as ∆[t] = t−U [t], where U [t] is
connections and yet clear distinctions from queueing delay. the most recent update time for the PoI. We assume that every
These key differences between AoI and queueing delay have user will report the real-time state information when he/she
sparked intense research in, e.g., real-time sampling and re- arrives at the PoI. Clearly, under a reshaped arrival process
mote estimation trade-off [4], [5], joint source-channel coding {A[t]}t≥0 , the AoI process {∆[t]}t≥0 evolves as follows:
exploitation [6], [7], caching [8], [9], optimization algorithms (
∆[t] + 1, if A[t] = 0;
for AoI minimization [10], [11], age-based scheduling [12], ∆[t + 1] = (1)
[13], just to name a few. However, research on AoI in mobile 0, if A[t] = 1.
crowd-learning remains in its infancy. The most related work In this paper, we consider both i.i.d. (independent and iden-
to this paper is [14], where the authors proposed a new tically distributed) and Markovian unshaped arrivals. We also
dynamic model that captures the most essential features of assume that the PoI serves exactly one user if there is any. As
many mobile crowd-learning systems with selfish users. Based a result, there is no queueing effect at the PoI. With the above
on this analytical model, they considered a linear reward system setting, a fundamental question is: Given a prediction
mechanism and investigated the AoI performance under selfish window of size w, how could we design an arrival reshaping
user behaviors measured by price-of-anarchy (PoA). We note policy to change the inter-arrival times of the users, so that the
that our work differs from [14] in the following key aspects: information freshness of the PoI can be improved? Answering
i) The model in [14] does not consider any predictions. In this question constitutes the rest of the paper.
comparison, our focus in this paper is to explore the impacts
of predictions in mobile crowd-learning; ii) While the goal IV. A RRIVAL R ESHAPING P OLICY D ESIGN
in [14] was to evaluate the AoI performance of the linear In this section, we take a first step to answering the
reward mechanism, the emphasis of this paper is to design an fundamental question in Section III by proposing an arrival
arrival reshaping policy based on predictions to improve AoI reshaping policy called “Periodic Equal-Spreading” (PES).
performance; iii) Unlike the model in [14] that only considered Towards this end, we first discuss the motivation and rationale
i.i.d. Bernoulli arrivals, we further consider a more challenging behind the PES policy in Section IV-A, which is followed
Markovian arrivals process. Because of these key differences, by the formal presentation of the general PES policy in
the results in this paper are all new. Section IV-B. Then, we will discuss the impact of a key
III. S YSTEM M ODEL AND P ROBLEM S TATEMENT parameter called “step size” on the performance of the PES
policy in Section IV-C.
Consider a mobile crowd-learning system with one PoI as
shown in Fig. 1. The PoI could represent, e.g., a road inter- A. Motivation and Rationale behind the Policy Design
section, a parking garage, a WiFi hot spot, a gas station, etc. Before formally stating our PES policy, it is insightful to
We consider a time-slotted system. In each time slot t, the PoI take a look at the rationale behind this policy. Our PES policy
holds some real-time state information (e.g., congestion level, is motivated by the fact that the periodic reshaped arrivals of
parking rate and space, gas price, etc.) that is time-varying and the users enable us to approximately sample the PoI evenly in
to be sampled by the arriving users. A service provider (i.e.,
a crowd-learning-based information/data analytics platform) 1 The impacts of prediction errors will be left for our future studies.

703
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 temporal domain in a spirit similar to the Round-Robin scheme The intuition of Algorithm 2 is that, in order to reshape the
in the spatial domain, which is known to be age-optimal in the predicted arrivals to be equally spread, we need to equalize
case with multiple PoIs (see [14]). Consider the single-PoI the inter-arrival times. If the number of arrivals ni and arrival
system shown in Fig. 1 with bursty arrivals. For example, the prediction window size w are given, the number of time slots
arrivals follow the pattern that every three consecutive arrivals with no arrivals is w − ni . Thus, one only needs to distribute
are followed by three time slots that have no arrivals, i.e., the these w − ni slots of no arrivals into ni + 1 groups. Let ki
arrival sequence is “111000111000 . . .”, where ‘1’ denotes that and bi be the quotient and remainder of (w − ni )/(ni + 1),
a user arrives at the PoI and ‘0’ denotes no user arrival. In i.e., w − ni = ki (n + 1 − bi ) + (ki + 1)b. In other words,
comparison, consider an alternative “even” arrival sequence (w − ni ) zeros could be partitioned into (n + 1 − bi ) zero-
“101010101010 . . .,” which has the same arrival rate. Suppose valued sequences of length ki and bi zero-valued sequences
that the initial age of the PoI is 0. The age evolution processes of length (ki + 1).
of both sequences are shown in Fig. 2. An Example of Equal Spreading: Consider a predicted ar-
We can see that the average and peak ages of the bursty- rival sequence “000011.” In this case, we have w = 6 and
arrival sequence are twice and three times as high as those of ni = 2, which entails bi = (6 − 2) mod (2 + 1) = 1
the even-arrival sequence, respectively. Indeed, it can be shown and ki = b(6 − 2)/(2 + 1)c = 1. According to Line 2
that the average and peak ages are both minimized when user of Algorithm 2, we generate two zero-valued sequences of
arrivals are equally-spaced between each other in the temporal length 1 (i.e., “0”) and one zero-valued sequences of length 2
domain. This insight inspires us to propose the PES policy in (i.e., “00”). Shuffling the zero-valued sequences uniformly at
Section IV-B, which spreads the foreseen arrivals within the random and inserting a “1” between every pair of adjacent
limited prediction window to generate nearly equally-spaced zero-valued sequences could yield “010100”, “010010” or
arrivals to decrease average and peak ages. “001010.” We can see that any of these reshaped sequences is
B. The Periodic Equal Spreading Policy more even than the original unshaped arrival sequence.
The basic idea of the PES policy is that, periodically, given In fact, it can be shown that the reshaped arrival sequence
an arrival sequence that is predicted within a window of size resulted from Algorithm 2 is the “most even” one in the sense
w into the future, the PES policy reshapes the arrivals in such that the lengths of the zero-valued sequences in ai produced
a way that the inter-arrival times are (nearly) equalized. The by Algorithm 2 have the minimum variance. We state this
PES policy is stated in Algorithms 1. insight as follows (proof details are relegated to Appendix A):

Algorithm 1: Periodic Equal-Spreading (PES) Policy . Proposition 1 (Most Even Reshaping). With two natural
numbers M and N such that M ≥ N , define an (N + 1)-
Initialization: partition of M as a set XN +1 , {X1 , . P
. . , XN +1 }, where all
1. Choose a step size value s ∈ {1, . . . , w}. Let i = 1. N +1
Xi ’s are natural numbers and satisfy i=1 Xi = M . Let
Main Loop: k = bM/N c and b = M mod N . Then, any (N + 1)-partition
2. In the i-th time-slot, observe the current time-slot and of M with (N +1−b) k-valued elements and b (k +1)-valued
predict the future w − 1 time-slots to obtain the vector elements, denoted as XN∗ +1 , has the minimum variance in all
(N + 1)-partitions.
âi that contains the sequence
 > in the i-
of arrivals foreseen
th time-slot, i.e., âi , Â[i], . . . , Â[i+w−1] ∈ {0, 1}w .
C. The Impact of Step Size
Let ni = kâi k1 , where k · k1 denotes the `1 norm.
3. Perform “equal spreading” on âi using Algorithm 2 to Given the PES policy, one important question immediately
obtain a reshaped arrival sequence ai . Let i = i + s and arises: How to pick a good step size (cf. Step 1 in Algo-
go to Step 2. rithm 1)? A closer look at the PES policy reveals that it
bears close resemblance to the classic MPC method (model
The “equal spreading” subroutine used in Algorithm 1 is predictive control, a.k.a. receding horizon control [15]) when
stated as follows: the step size s = 1. Specifically, the controller in the MPC
Algorithm 2: Equally spreading the predicted arrivals in âi . method computes control/optimization decisions over a finite
future time horizon, but only implements the current time-slot
1. Given the prediction window size w and the length of the and then computes control/optimization decisions in the next
predicted arrival sequence ni = kâi k1 , compute bi = (w − time-slot again. It has been widely observed that, although
ni ) mod (ni + 1) and ki = b(w − ni )/(ni + 1)c, where being a heuristic, the MPC method has excellent empirical
bxc denotes the maximum integer that is not greater than performance [16]. Therefore, one may tend to choose s = 1
the real number x. in our PES policy. Surprisingly, in what follows, we will show
2. Generate (ni + 1 − bi ) zero-valued sequences of length ki that the “MPC step size” (s = 1) is a poor choice for our PES
and bi zero-valued sequences of length ki + 1. policy in terms of the AoI performance.
3. Shuffle these sequences uniformly at random and insert
An Example of the MPC Fallacy: Suppose that the original
a “1” element (i.e., an arrival) between every pair of
arrival sequence is “001000”, and w = 3. When s = 1, the
consecutive intervals. Return the reshaped sequence as ai .
reshaping under MPC method is shown in Fig. 3, where bold

704
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 3 4
(unshaped) (current) future
Bursty
2.5 peak: 3 time−slot: 0 3.5
avg.: 1
(current) future 3 Reshaped
2 time−slot: 0 2.5
Periodic
(current)

Age
Age

peak: 1
1.5
avg.: 0.5
future 2
time−slot: 1
1.5 Unshaped
1
(current) future 1
0.5
time−slot: 2
0.5
(current) future
0
0 5 10 15 20 25 30
time−slot: 3 0
0 1 2 3 4 5 6 7
Time Time

Fig. 2: Age performance with bursty and Fig. 3: An example of the PES policy Fig. 4: Age performance with MPC
periodic arrivals. with step size s = 1. when s = 1 and w = 3.
our experiments. Hence, in what follows, we set s = w for
5
analytical tractability.
4 V. P ERFORMANCE E VALUATION
In this section, we analyze the age performances of PES
Average Age

3 policy under both i.i.d. Bernoulli arrivals and Markovian

i.i.d. Arrivals arrivals. For this purpose, we introduce two key notations,
2
Markovian Arrivals Di and Xij , where Di denotes the number of time-slots from
the time of the last seen arrival to the beginning of the i-th
1 prediction window, and Xij is the length of the j-th zero-
valued subsequence within the i-th prediction window. Fig. 6
0 is an illustration of Di and Xij .
0 0.2 0.4 0.6 0.8 1
Step Size/Window Size wi−1 wi
Fig. 5: The effects of step size. future

Di Xi1 Xi2 Xi3 Xi4

line segments and dotted bars denote prediction windows and
arrivals, respectively. We can see that the PES policy places Fig. 6: An illustration of Di and Xij .
the future arrival at the center of the current prediction window
when s = 1. This effectively creates a “pushing” effect, which A. Independent and Identically Distributed Bernoulli Arrivals
delays the sampling time of the PoI and leads to worse AoI In this subsection, we consider the simpler case with i.i.d.
performance. The comparison of the age performance between Bernoulli arrivals. The results of the i.i.d. Bernoulli arrivals
the unshaped and the reshaped arrival sequences is shown in are not only interesting in their own rights, they also serve as
Fig. 4. It is obvious that the average age and peak age of the a foundation for the more complex Markovian arrival case. We
reshaped arrival sequence are larger than that of the unshaped first analyze the age performance without using any reshaping
arrival sequence when s = 1. policy as a baseline.
Now that knowing s = 1 is not preferable, it remains to Theorem 1 (Age of Unshaped i.i.d. Bernoulli Arrivals).
choose an optimal step size s ∈ {2, . . . , w}. Unfortunately, Without any reshaping policy, the expected average age over
determining an optimal step size is hard. Particularly, when an i.i.d. Bernoulli arrival sequence with arrival probability p
s < w, due to the tight coupling and long-range dependence can be computed as:
between prediction windows, it is intractable to characterize 1
the effect of the step size on AoI in a closed-form expression. ¯ =
E[∆] − 1. (2)
p
Fortunately, extensive experiments show that there exists a
“phase transition” with respect to the step size. The effects Proof. First, we introduce a lemma (see Lemma 1 of [17]),
of step size on an i.i.d. arrival sequence with p = 0.3 and a which is useful to prove the stated result:
two-state Markovian arrival sequence with transition rate being Lemma 1. For any arrival process for which the steady-state
0.3 are shown in Fig. 5. The window size in the previous two distribution exists, it holds that
examples are 50. We can see that when s/w < 0.1, the average 2
¯ = 1 E[X ] − 1 ,
 
performance is poor. However, once the step size is sufficiently E[∆] (3)
large (s/w ≥ 0.1), the age performance is insensitive with 2 E[X]
respect to s. This phenomenon occurs consistently in all of where X is the length of the inter-arrival time.

705
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 j k
When the arrivals follow the i.i.d. Bernoulli-p distribution, Since ki = w−n i w−ni
ni +1 , we have ni +1 − 1 < ki ≤
w−ni
ni +1 . Hence,
X is a geometrically distributed random variable. Thus, we f (ki ) ≤ f ( w−n 1
ni +1 − 2 ). Then we have:
i

have E[X] = p1 and E[X 2 ] = Var[X] + E[X]2 = 1−p 1

p2 + p2 =
ni +1
2−p ¯ 1 1 X w+1 w−ni 1
p2 . Plugging them into (3), we have E[∆] = p − 1. Xij (Xij + 1) ≤ + (ni +1). (8)
2 j=1 2 ni +1 8
Due to the complex coupling between the arrivals in differ-
ent prediction windows, it is hard to analyze the exact mean Note that we shuffle the inter-arrivals uniformly at random
age performance even under the i.i.d. arrival case. Instead, in our algorithm, the expectation of Xij resulted from the
we provide the following tight upper bound (which is also shuffling is w−n
ni +1 . When s = w, Di is independent of Xi .
i 1

validated through simulations). Hence, it follows that:

hw − n i
i
Theorem 2 (Age Upper Bound of the PES Policy: i.i.d. E[Di Xi1 ] = E[Di ]E[Xi1 ] = E[Di ]E . (9)
Arrivals). With PES policy and s = w, the upper bound of the ni + 1
expected average age over an i.i.d. Bernoulli arrival sequence Combining (8) and (9), we further have:
with arrival probability p can be computed as: w+1 h w−ni i h w−n i 1
i
E[∆wi ]≤ E +E[Di ]E + (E[ni ]+1).
PI 2 ni +1 ni +1 8
1−(1−p)w+1 1
  
¯ w+1 i=1 E[Di ] (10)
E[∆] ≤ − +lim sup
pw w 2 I→∞ I Note that,
wp+1 i−1
+ , (4) X n
8w E[Di ] = P (ni−1 , ni−2 , · · · , ni−k = 0, ni−1−k > 0) kw
where k=0
h w−n io
i
i−1 +E ni > 0 +P (ni−1 , ni−2 , · · · , n0 = 0)(iw).
X n 1 ni +1
E[Di ] = kw−1+ [1−(1−p)w ]
(1−p)wk (11)
p (5)
k=0
o If the arrivals are i.i.d., then ni and nj are independent if
−w(1−p)w +(1−p)wi (iw), ∀i ≥ 1. i 6= j. It follows that P (ni−1 = 0) = (1 − p)w , P (ni−1 ≥
0) = 1 − (1 − p)w , and E[Xij |ni > 0] = E[ w−n ni +1 |ni > 0],
i

Proof. If the interval length between two arrivals is X, then ∀i, j. Then, plugging the above back to (11), we have:
the age sequence during this interval is 0, 1, 2, · · · , X. It
i−1
follows that the age sum of this interval is X(X + 1)/2. We X  h w−n
i
i
E[Di ] = (1−p)wk [1− (1−p)w ] kw+E ni > 0
then obtain the age sum of a window by adding the age sum ni +1
k=0
of each interval within that window. Note that there may be an
initial age at the beginning of each window, which only affects + (1 − p)wi (iw). (12)
the age in the very first interval within that window. Recall From (10) and (12), we can see that, to obtain an upper
that Di is the distance from the last arrival to the beginning bound of E[∆w w−ni
i ], we only need to calculate E[ni ], E[ ni +1 ],
of the i-th window, the initial age of the i-th window is Di . w−ni
and E[ ni +1 |ni > 0]. Towards this end, note that, s = w and
Then the age sum of the i-th window can be computed as: the arrivals are i.i.d., the following equalities hold:
ni +1
1 X
∆w
i = X j (X j + 1) + Di Xi1 , ∀i ≥ 1, (6) E[ni ] = wp, (13)
2 j=1 i i
h 1 i X w
1
E = P (ni = k)
where ni is the number of arrivals in the i-th window, Xij is the ni +1 k+1
k=0
length of the j-th interval of the i-th window. For convenience, w  
we will omit “∀i ≥ 1” for the following equations.
X 1 w k
= p (1 − p)w−k
As described in Algorithm 2, w − ni = ki (ni + 1) + bi = k+1 k
k=0
ki (ni + 1 − bi ) + (ki + 1)bi , which implies that there are 1
ni + 1 − bi intervals of length ki and bi intervals of length = [1 − (1 − p)w+1 ], (14)
(w + 1)p
ki + 1 in the i-th window. Thus, we have: h 1 w
i X 1 P (ni = k)
E ni > 0 =
1X
n +1
i
(ni +1−bi )ki (ki +1)+bi (ki +1)(ki +2)] ni +1 k + 1 1 − P (ni = 0)
k=1
X j (X j +1) =
2 j=1 i i 2 1 h 1−(1−p)w+1 w
i
= −(1−p) . (15)
(ni +1)(ki +1) 2(w−ni )
h i 1−(1−p)w (w+1)p
= −ki . (7) It then follows that,
2 ni +1
hw − n i hw+1 i 1
We denote the right hand side of (7) as f (ki ), which is a E
i
=E − 1 = [1 − (1 − p)w+1 ]−1, (16)
quadratic function with the axis of symmetry being w−n 1 ni + 1 ni + 1 p
ni +1 − 2 .
i

706
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 p1 Lemma 2. The matrix of the second moments of the first
passage time can be computed as:
1-p1 0 1 1-p2
(2)
Md = 2Md (ΠM )d − Md , (22)
p2
where Π is a 2 × 2 matrix with each row being the stationary
Fig. 7: The Gilbert-Elliot model. distribution π, M = [mij ] is the matrix of the first moments
of the first passage times, Md = [δij mij ] (δij = 1, if i = j,
0, otherwise) is a diagonal matrix with elements being the
(2) (2)
diagonal elements of M and Md = [δij mij ].
h w−n i h w+1 i
i
E ni > 0 = E −1 ni > 0 (2)
ni +1 ni +1 Next we are going to calculate m11 with Lemma 2. For
w+1 h 1−(1−p)w+1 w
i stationary distribution π, we have
= −(1−p) −1. (17)
1− (1−p)w (w+1)p
π = πP, (23)
Plugging (17) into (12) yields (5). Then, plug (13) and (16)
into equation (10), we obtain π0 + π1 = 1. (24)
 1−(1−p)w+1  w+1  wp+1
E[∆w i ]≤ −1 +E[Di ] + . (18) With (20), (23) and (24), we can derive:
p 2 8 h p
2 p1 i
The upper bound of overall average age can be calculated π , [π0 , π1 ] = , . (25)
p1 + p2 p1 + p2
as: PI w
¯ ≤ lim sup i=1 E[∆i ] .
E[∆] (19) It then follows that:
I→∞ wI    p2 p1 
π π1
Therefore, with (18) and (19), we arrive at (4). This com- Π, 0 = p1p+p 2 p1 +p2
p1 , (26)
π0 π1 2
p1 +p2 p1 +p2
pletes the proof.
1 p1 + p2 1 p1 + p2
B. Markovian Arrivals m00 = = , m11 = = , (27)
π0 p2 π1 p1
For the Markovian arrivals, we consider the Gilbert-Elliot
model, which is a two state Markov chain. As shown in Fig. m01 = p01 + (1 + m01 )p00 , m10 = p10 + (1 + m10 )p11 . (28)
7, States 0 and 1 represent “no arrival” and “an arrival has
occurred,” respectively. Also, p1 and p2 are the state transition With (27) and (28), we have:
probabilities, i.e., the transition matrix P is: "
p1 +p2
#
1
"
p1 +p2
#
p2 p1 p2 0
M= , Md = . (29)
 
1 − p1 p1 1 p1 +p2
0 p1 +p2
P = . (20) p2 p1 p1
p2 1 − p2
Similar to Section V-A, we first establish the following Plugging (26) and (29) into (22), we then derive:
result for unshaped Markovian arrivals as a baseline. " p +p
+ 2p
#
(2)
1
p2
2
p22
1
0
Theorem 3 (Age of Unshaped Markovian Arrivals). Without Md = p1 +p2 , (30)
0 p1 + 2p
p2
2
1
any reshaping policy, the expected average age over a Marko-
vian arrival sequence following the Gilbert-Elliot model can which implies that m11 =
(2) p1 +p2
+ 2p 2
. Then with Lemma 1,
p1 p2
be computed as: (2)
1
m
¯ = 1 ( 11 −1) = p2
p2 E[∆] 2 m11 p1 (p1+p2 ) . This completes the proof.
¯ =
E[∆] . (21)
p1 (p1 + p2 )
Next, we state the average age upper bound of the PES
Proof. Similar to the proof of Theorem 1, we will use the policy under Markovian arrivals.
Lemma 1 of [17]. Thus we need to know the first and second
moments of the inter-arrival times. For Markovian arrivals, the Theorem 4 (Age Upper Bound of the PES Policy: Markovian
inter-arrival time is equivalent to the recurrence time for State Arrivals). With PES policy and s = w, the upper bound of
1. Let us denote the stationary distribution vector π as [π0 , π1 ], the expected average age over a Markovian arrival sequence
where πi (i = 0, 1) is the stationary probability of being at (at steady state) following the Gilbert-Elliot model can be
State i. We also use mij to denote the expected first passage computed as:
(2)
time from State i to State j, and mij denotes its second
¯ ≤ w+1 E w−ni + E[ni ]+1
h i
moment. Then the first moment of the recurrence time for E[∆]
(2)
2w ni +1 8w
State 1 (m11 ) can be represented as 1/π1 . To calculate m11 , r h I q (31)
w−ni 2 i 1 X
we need the following lemma (see [18, Corollary 2.4.2]). + E lim sup E[Di2 ],
ni +1 I→∞ Iw i=1

707
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 20 1.6 10
Theoretical values
Experiment results 1.5
8
15 1.4 w = 80

Average Age

Average Age
Average age

1.3 w = 60
6
10 1.2 w = 40 No reshaping
w = 20 (w = 1)
1.1 4
Upper Bound
5 1 Experiment Results
2
0.9

0 0.8 0
0.1 0.2 0.3 0.4 0.5 0 20 40 60 80 100 0.1 0.2 0.3 0.4 0.5
p Window Size p

Fig. 8: AoI performance for different Fig. 9: AoI performance under the PES Fig. 10: AoI performance comparison
i.i.d. Bernoulli arrivals without any re- policy with respect to window size for for i.i.d. Bernoulli arrivals with varying
shaping policy. i.i.d. Bernoulli arrivals (p = 0.4). prediction window sizes.
5 4.5 4
Theoretical values
Experiment results 4 3.5
4 No reshaping
3.5 3 (w = 1)

Average Age
Average age

Average Age

3 3 2.5
w = 10
2.5 Upper Bound 2 w = 70
w = 30 w = 50
2 1.5
2
Experiment Results 1
1.5
1
1 0.5

0 0.5 0
0.1 0.2 0.3 0.4 0.5 0 20 40 60 80 100 0.1 0.2 0.3 0.4 0.5
p 2 (p1 = 0.2) Window Size p 2 (p1 = 0.2)

Fig. 11: AoI performance for different Fig. 12: AoI performance under the PES Fig. 13: AoI performance comparison for
Markovian arrivals without any reshap- policy with respect to window size for Markovian arrivals with varying predic-
ing policy. Markovian arrivals (p1 = 0.2, p2 = 0.3). tion window sizes.
where VI. N UMERICAL R ESULTS
i−1
X n In this section, we conduct simulations to verify the age
E[Di2 ] = P (ni−1 , ni−2 , · · · , ni−k = 0, ni−1−k > 0) (kw)2 performance under the PES policy for i.i.d. Bernoulli arrivals
k=0 and Markovian arrivals in a single-server system. In the
h w−n i h w−n 2 io
+ 2kwE
i
ni > 0 + E
i
ni > 0 following simulations, we generate 1, 000 arrival sequences
ni +1 ni +1 of length 100, 000 for each trial uniformly at random.
+P (ni−1 , ni−2 , · · · , n0 = 0)(iw)2 . (32)
A. Independent and Identically Distributed Bernoulli Arrivals
Proof. Similar to the proof of Theorem 2, Eqs. (6) to (8) also
First, to confirm the results in Theorem 1, we evaluate the
hold for Markovian arrivals. However, for Markovian arrivals,
average age performance without reshaping and the results
Di is dependent of Xi1 , which means that Eq. (9) does not
are shown in Fig. 8. We can see that the experimental results
hold. To address this issue, by Cauchy-Schwarz inequality,
perfectly match our theoretical predictions in Theorem 1.
we have:
r Then, we evaluate AoI performance under the PES policy
q h w−n 2 i
1 i with respect to window size for i.i.d. Bernoulli arrivals to
E[Di Xi ] ≤ E[Di ]E[(Xi ) ] = E[Di2 ]E
2 1 2 .
ni +1 verify Theorem 2 and the results are shown in Fig. 9 (we
(33) only show the case with p = 0.4 due to space limitation).
Thus, combining (8) and (33) yields: Out experimental results show that the upper bound stated in
w+1 h w−ni i 1 Theorem 2 is tight for window sizes ranging from one to 100.
E[∆wi ]≤ E + (E[ni ]+1) Finally, we evaluate the AoI performance under the PES
2 ni +1 8
r h w−n 2 i (34) policy with respect to the prediction window size (w = 1
2 i
+ E[Di ]E . corresponds to the no-reshaping case). As shown in Fig. 10,
ni +1
under the PES policy, the AoI performance is significantly
By the definition of Di , we have the stated result in (32). better compared to that of the no-reshaping case. Also, the AoI
Lastly, plugging (34) into (19) leads to the final result stated performance improves as w gets large, which makes intuitive
in (31). This completes the proof. sense because larger w implies better prediction. However, we

708
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.
 also note a diminishing return effect: the AoI improvement [5] X. Gao, E. Akyol, and T. Basar, “Optimal estimation with limited
becomes increasingly marginal as w gets large. measurements and noisy communication,” in Proc. IEEE CDC, Osaka,
Japan, December 2015, pp. 1775–1780.
[6] E. T. Ceran, D. Gunduz, and A. Gyorgy. (2017, October) Average age
B. Markovian Arrivals of information with hybrid arq under a resource constraint. [Online].
Available: https://arxiv.org/abs/1710.04971v1
First, to confirm the results in Theorem 3, we evaluate the [7] R. D. Yates, E. Najm, E. Soljanin, and J. Zhong, “Timely updates over
average age performance without reshaping and the results are an erasure channel,” in Proc. IEEE ISIT, 2017.
shown in Fig. 11. Again, the experimental results perfectly [8] C. Kam, S. Kompella, G. Nguyen, J. Wieselthier, and A. Ephremides,
“Information freshness and popularity in mobile caching,” in Proc. IEEE
match our theoretical predictions in Theorem 3. ISIT, June 2017, pp. 136–140.
Then, we evaluate AoI performance under the PES policy [9] R. D. Yates, P. Ciblat, A. Yener, and M. Wigger, “Age-optimal con-
with respect to window size for Markovian arrivals to verify strained cache updating,” in Proc. IEEE ISIT, June 2017, pp. 141–145.
[10] Y. Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff,
Theorem 4 and the results are shown in Fig. 12 (we only show “Update or wait: How to keep your data fresh,” IEEE Transactions on
the case with p1 = 0.2, p2 = 0.3 due to space limitation). Information Theory, vol. 63, no. 11, pp. 7492–7508, November 2017.
Out experimental results show that the upper bound stated [11] S. K. Kaul and R. D. Yates, “Status updates over unreliable multiaccess
channels,” in Proc. IEEE ISIT, 2017, pp. 331–335.
in Theorem 4 is valid. Moreover, the upper bound becomes [12] B. Li, A. Eryilmaz, and R. Srikant, “On the universality of age-based
sharper as w increases. We note that the looseness of the upper scheduling in wireless networks,” in Proc. IEEE INFOCOM, Kowloon,
bound for small w values is mainly due to the approximation Hong Kong, April 2015, pp. 1302–1310.
[13] N. B. Lakshminarayana, J. Lee, and H. Kim, “Age based scheduling
error of the Cauchy-Schwarz inequality. for asymmetric multiprocessors,” in Conference on High Performance
Finally, we evaluate the AoI performance under the PES Computing Networking, Storage and Analysis, Portland, OR, USA,
policy with respect to the prediction window size (w = 1 cor- November 2009.
[14] B. Li and J. Liu, “Can we achieve fresh information with selfish users in
responds to the no-reshaping case). As shown in Fig. 13, with mobile crowd-learning?” in International Symposium on Modeling and
reshaping, the AoI performance gain is even more pronounced Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), 2019.
compared to that of the i.i.d. Bernoulli arrival cases. Again, the [15] W.-H. Kwon and S. Han, Receding Horizon Control, 01 2005.
[16] J. Rossiter, Model-based Predictive Control-a Practical Approach, 01
AoI performance improves as w gets large in the Markovian 2003.
arrival cases. Interestingly, the same diminishing return effect [17] B. Li, R. Li, and A. Eryilmaz, “Throughput-optimal scheduling design
also occurs for Markovian arrivals. with regular service guarantees in wireless networks,” IEEE/ACM Trans-
actions on Networking, vol. 23, 05 2014.
[18] J. J. Hunter, “Variances of first passage times in a markov
VII. C ONCLUSION chain with applications to mixing times,” Linear Algebra and its
Applications, vol. 429, no. 5, pp. 1135 – 1162, 2008, special
In this paper, we strived to understand the impacts of Issue devoted to selected papers presented at the 13th Conference
predictions on information freshness over a single-PoI system. of the International Linear Algebra Society. [Online]. Available:
To answer this question, we first introduced a single-PoI http://www.sciencedirect.com/science/article/pii/S0024379507002832
system model that takes into account the essential features A PPENDIX A
of predictive mobile crowd-learning. Based on this model P ROOF OF P ROPOSITION 1
and motivated by the fact that periodic arrivals have better
Proof. Note that, M = (N +1)k+b = (N +1−b)k+b(k+1).
AoI performance than bursty arrivals, we proposed an arrival
Thus XN∗ +1 is a (N + 1)-partition of M . Let YN +1 and
reshaping policy called ”periodic equal spreading” (PES) to
ZN +1 be the partitions of k(N + 1) and b, respectively.
generate nearly equally-spaced arrivals to decrease average
It is evident that partitioning M into (N + 1) parts is
and peak ages. To analyze the AoI performance of the PES
equivalent to partitioning k(N + 1) and b into (N + 1)
policy, we considered two types of arrivals: i.i.d. Bernoulli and
parts. Then we have Xi = Yi + Zi , i = 1, 2, . . . , N +
Markovian arrivals. For each type of arrivals, we first derived a
1. It follows that Var(XN +1 ) = Var(YN +1 + ZN +1 ) =
closed-form expression for the average age without reshaping.
Var(YN +1 ) + Var(ZN +1 ) + 2Cov(YN +1 , ZN +1 ). Note that
Then we established upper bounds for the average age under
Cov(YN +1 , ZN +1 ) = 0 since YN +1 and ZN +1 are indepen-
the PES policy. Numerical results match our analysis well. We
dent. Thus, Var(XN +1 ) = Var(YN +1 ) + Var(ZN +1 ). Let CX
know that the research on AoI in predictive mobile crowd-
be the collection of all possible XN +1 ’s, and CY , CZ be the
learning remains an under-explored area and many problems
collections of all possible YN +1 ’s and ZN +1 ’s, respectively.
are still wide open. Future directions include extensions to
Then we have:
multi-PoI systems, consideration of prediction errors, and fur-
ther predictions on real-time PoI state information processes. min Var(XN +1 ) = min (Var(YN +1 ) + Var(ZN +1 ))
XN +1 ∈CX YN +1 ∈CY ,
ZN +1 ∈CZ
R EFERENCES
= min Var(YN +1 ) + min Var(ZN +1 ).
YN +1 ∈CY ZN +1 ∈CZ
[1] GasBuddy Mobile App. [Online]. Available: https://www.gasbuddy.com/
[2] Waze Mobile App. [Online]. Available: https://www.waze.com/ Then the minimum variance of YN +1 can be achieved when
[3] WiFi Finder Connect Internet, Mobile App. [On- Yi = k, ∀i, in which case Var(YN +1 ) = 0. In addition, since
line]. Available: https://itunes.apple.com/us/app/wifi-finder-connect-
internet/id1011519183?mt=8 b < N + 1, the minimum variance of ZN +1 can be achieved
[4] Y. Sun, Y. Polyanskiy, and E. Uysal-Biyikoglu, “Remote estimation of when there are b 1’s and (N − b) 0’s. Thus, XN∗ +1 achieves
the wiener process over a channel with random delay,” in Proc. IEEE the minimum variance.
ISIT, June 2017, pp. 321–325.

709
Authorized licensed use limited to: The Chinese University of Hong Kong CUHK(Shenzhen). Downloaded on June 27,2022 at 10:50:20 UTC from IEEE Xplore. Restrictions apply.