[Profile picture of Ruben Verborgh]

Ruben Verborgh

Comparing a Polling and Push-Based Approach for Live Open Data Interfaces

Brecht Van de Vyvere, Pieter Colpaert, and Ruben Verborgh

There are two mechanisms for publishing live changing resources on the Web: a client can pull the latest state of a resource or the server pushes updates to the client. In the state of the art, it is clear that pushing delivers a lower latency compared to pulling, however, this has not been tested for an Open Data usage scenario where 15k clients are not an exception. Also, there are no general guidelines when to use a polling or push-based approach for publishing live changing resources on the Web. We performed (i) a field report of live Open datasets on the European and U.S. Open Data portal and (ii) a benchmark between HTTP polling and Server-Sent Events (SSE) under a load of 25k clients. In this article, we compare the scalability and latency of updates on the client between polling and pushing. For the scenario where users want to receive an update as fast as possible, we found that SSE excels above polling in three aspects: lower CPU usage on the server, lower latency on the client and more than double the number of clients that can be served. However, considering that users can perceive a certain maximum latency on the client (MAL) of an update acceptable, we describe in this article at which MAL point a polling interface can be able to serve a higher number of clients than pushing. Open Data publishers can use these insights to determine which mechanism is the most cost-effective for the usage scenario they foresee of their live updating resources on the Web.

full text BibTeX other citation formats

Published in 2020 in Proceedings of the 20th International Conference on Web Engineering.

Keywords:

Read this article online

Cite this article in your work

Cite this article easily using its BibTeX entry: 

@inproceedings{vandevyvere_icwe_2020,
  author = {Van de Vyvere, Brecht and Colpaert, Pieter and Verborgh, Ruben},
  title = {Comparing a Polling and Push-Based Approach for Live {Open Data} Interfaces},
  booktitle = {Proceedings of the 20th International Conference on Web Engineering},
  editor = {Bielikova, Maria and Mikkonen, Tommi and Pautasso, Cesare},
  year = 2020,
  month = jun,
  pages = {87--101},
  volume = 12128,
  series = {Lecture Notes in Computer Science},
  publisher = {Springer},
  isbn = {978-3-030-50578-3},
  doi = {10.1007/978-3-030-50578-3_7},
  url = {https://brechtvdv.github.io/Article-Live-Open-Data-Interfaces/},
}

Alternatively, pick a reference of your choice below:

ACM
Brecht Van de Vyvere, Pieter Colpaert, and Ruben Verborgh. 2020. Comparing a Polling and Push-Based Approach for Live Open Data Interfaces. In Proceedings of the 20th International Conference on Web Engineering (Lecture Notes in Computer Science), Springer, 87–101.
APA
Van de Vyvere, B., Colpaert, P., & Verborgh, R. (2020). Comparing a Polling and Push-Based Approach for Live Open Data Interfaces. In M. Bielikova, T. Mikkonen, & C. Pautasso (Eds.), Proceedings of the 20th International Conference on Web Engineering (Vol. 12128, pp. 87–101). Springer.
IEEE
B. Van de Vyvere, P. Colpaert, and R. Verborgh, “Comparing a Polling and Push-Based Approach for Live Open Data Interfaces,” in Proceedings of the 20th International Conference on Web Engineering, 2020, vol. 12128, pp. 87–101.
LNCS
Van de Vyvere, B., Colpaert, P., Verborgh, R.: Comparing a Polling and Push-Based Approach for Live Open Data Interfaces. In: Bielikova, M., Mikkonen, T., and Pautasso, C. (eds.) Proceedings of the 20th International Conference on Web Engineering. pp. 87–101. Springer (2020).
MLA
Van de Vyvere, Brecht, et al. “Comparing a Polling and Push-Based Approach for Live Open Data Interfaces.” Proceedings of the 20th International Conference on Web Engineering, edited by Maria Bielikova et al., vol. 12128, Springer, 2020, pp. 87–101.

Discuss this article