Image2Weather: A Large-Scale Image Dataset for Weather Property Estimation

[pdf-embedder url=”https://wellapets.com/wp-content/uploads/2019/06/Image2Weather-A-Large-Scale-Image-Dataset-for.pdf” title=”Image2Weather A Large-Scale Image Dataset for”]

Abstract—To facilitate weather property estimation from images,
a large-scale image dataset associated with rich weather
information is developed. Based on the taken time and geographical
information of an image, we associate it with weather
properties obtained from a weather forecast website. Through
data filtering like indoor/outdoor classification and sky region
detection, a clean and large-scale image-weather dataset (named
Image2Weather dataset) consisting of more than 180,000 photos
is built to promote related researches. In addition to reporting
statistical characteristics of this dataset, we investigate the relationship
between several visual features and weather properties,
which then serve as the foundation of interesting applications
like weather type classification and temperature estimation. We
show effectiveness of weather property estimation based on the
Image2Weather dataset, and discuss how it can be leveraged to
facilitate related studies in the future.

INTRODUCTION
Estimating image properties from visual content is a fundamental
step of various computer vision studies. For example,
estimating image scene labels [1] [2] facilitates image
browsing and retrieval, and recognizing whether images were
captured indoors or outdoors [3] facilitates place recognition.
Recently, estimating geographic information from images [4]
attracts much attention because various potential applications
can be expected. In this paper, we advocate a study of an image
property that affects visual appearance of images and is well
perceived by human beings, but has attracted little research
attention for a long time: weather information.
By analyzing geographical or weather information of usergenerated
images shared on the web, we could unveil characteristics
in the real world from images available in the cyberspace.
Comparing with image’s geographical information,
weather properties keep changing even images were captured
at the same place. We thus think that weather variations across
time provide richer information and may give impact to wider
fields. For example, by estimating weather information from
images uploaded by users, the population’s cameras can be
viewed as weather sensors, and fine-grained weather monitoring
can be achieved. Coupling estimated weather information
with time/geographical information, explicit or implicit human
behaviors can be discovered. For example, more people travel
(and thus more photos taken) on weekends if it is sunny,
and some places are especially attractive if the temperature is
under −5◦C. Weather properties can also serve as important
prior information for many computer vision applications, e.g.,
object detection/recognition, scene categorization, and image
retrieval. Examples in Fig. 1 show that Eiffel Tower has drastically
different visual appearances in different weather conditions,
which draws significant challenges on object/landmark
recognition. Once weather properties can be estimated, an object
detector/recognizer can adapt its parameters for different
weather conditions, so that influence of visual variations can
be reduced.
Although estimating weather properties from images brings
many research potentials, related studies are just at an infant
stage and emerging research ideas have not been well
exchanged due to lack of common benchmark and baseline
experimental studies. Our goal in this paper is to build a largescale
image dataset where images were captured by amateur
photographers spanning across the Europe, and each image
in the dataset is associated with rich weather information. To
demonstrate that estimating weather properties from consumer
photos1 is a doable computer vision research, in this work
we particularly focus on: (1) How to collect a large-scale
image collection associated with weather information and
other useful metadata? (2) What explicit/implicit knowledge
is embedded by such cross-platform image data? (3) What
kind of applications can be benefited by the estimated weather
properties?
The rest of this paper is organized as follows. Related
works are reviewed in Section II. In Section III, we describe
how to crawl weather information from a web-based
weather platform, i.e., Weather Underground2, based on an
existing large-scale image collection that was collected from
Flickr3, i.e., the European City 1M (EC1M) dataset [5]. In
Section IV, we will show interesting statistics derived from the
collected dataset. Correlation between metadata/visual features
and weather information will be demonstrated. Section V
describes potential applications based on the built dataset,
giving clues for future weather-related researches. Summary
and future works are given in Section VI.