Bootstrap vs Custom CSS

Recently I have this idea to create a small web app to manage list of tasks with sub-tasks. Coming from system programming field to writing web application and looking to dozen web frameworks there was the obvious question – what to use?

tasks
Using Flexbox to layout tasks elements

After some time of research and trying things the choice falls on Bootstrap framework, as it’s actively maintained and community is large in case of trouble getting your code to work. Another option is to use pure CSS + JavaScript to style and write logic to make similar components  included in Bootstrap out of the box and much more.

It is tempting to use first option as you see all building blocks in Bootstrap ready to use and imagine how easy it would be building web app using their layout classes and different components. On the other hand you stay alone with a ton of CSS documentation plus your own implementation with additional third-party libraries, not to mention hacking CSS in order to overcome different kind of issues related to specific version of web browser.

Some time ago I’ve chosen Bootstrap over custom development for another project. So far so good until I hit the wall when I needed to do something that was not built-in into Bootstrap classes and components as well as not already answered on blogs/forums. So, extending already existing classes was a nightmare.

This time I’ve tried to do custom CSS and to learn how to do things in a right way. Here is brief summary what you need to learn in order to have base understanding to move further into styling complex web application with large number of components.

  1. First of all to have introduction what CSS is all about. Sure, you’ve read some CSS and know something about it, but this introduction is more to organize your knowledge about selectors, pseudo-classes and elements, box model, units in order to switch from hacking stuff together and make it work to understanding why it works like this and how to fit it to your needs.
  2. CSS Layout – that’s important and the hard one. If you want to arrange elements on webpage in specific order you need to understand how to do this correctly without hard-coding left/right/top/bottom values. I recommend to take a glance at floats, positioning and then invest some time on deep understanding how Flexbox and Grids work. This is powerful concept to grasp as this will decrease the time spent on figuring out why does this element is not positioned correctly or not showing at all.
  3. Media Queries – you need to think about different devices, it’s not only your desktop web browser window anymore. There are different kinds of output devices and media queries is an easy way to remove/replace/add new styles based on different input parameters.

Conclusion

So, after you grind all documentation along with writing simple web applications in JavaScript + CSS, supporting different devices, fixing bugs in different web browser versions – now, you have understanding how Bootstrap or similar web framework works by encapsulating all functionality and allows you to forget about this for some period of time.

Now, after you understood how this works under the hood, the question what to use is resolved by themselves. If you need it fast – you can use Web frameworks, if you need lightweight solutions – no problem, just use custom approach or combine Web framework with custom CSS and JavaScript, but the important part is that you have understating how it works and how to fix it on your own.

Axis Order – What number comes first longitude or latitude?

What number comes first latitude or longitude? The answer is pretty simple – it depends on what software/library you are using.

Here is compiled list of popular desktop/web/formats/programming libraries and the axis order they are using.

This list was compiled after some time of using those packages and of course fixing bugs when you passing tuple of coordinates in latitude, longitude format to package which is using (surprise!) longitude, latitude order, that’s was a fun night!

Latitude, Longitude Longitude, Latitude
GeoRSS PostGIS
Leaflet GeoJSON
Google Maps API KML
Apple MapKit Shapefile
Bing Maps API WKT
OpenStreetMap WKB
HERE maps OpenLayers
ArcGIS API
Mapbox API
Redis
Mapnik
QGIS
SpatiaLite
MySQL spatial
Oracle spatial

In case you found something wrong or want to add new entry to the table above, just leave comment and I fix/add entry.

 

The Definitive Guide to Google Maps Markers

Here is the list of most common topics that everybody who’s working with markers in Google Maps is facing from time to time. The list contains simple topics like adding/removing markers, but that’s for the sake of having all subjects in one place. In each section there is a code snippets with optional link to live preview.

Contents

How to add marker

Adding new Marker is easy, all you need to do is to call Marker constructor with additional options, and the mandatory parameter is a map, where you want to put a marker.

function add_marker(gmap, coordinates, text) {
   var marker = new google.maps.Marker({
        position: coordinates,
        map: gmap,
        label: { text: text }
    });

   return marker;
}

Live preview or view source code

How to remove marker

There are two ways to remove marker from a map. The first one is to hide it using Maker.setVisible method and the second is to set the value of map where you want to put the marker to null using Marker.setMap method. The difference between two methods is that the setVisible just hide marker and preserves resources and the setMap removes marker and releases allocated resources.

If you want to show/hide small number of markers you can use setVisible methodbut imagine you have 10.000 of markers and you want periodically hide them. In this case using setVisible method maybe not such a good idea, as all hidden markers hanging on the map and use resources, so in this case using setMap method maybe a better choice.

function add_marker(gmap, coordinates, text) {
    var marker = new google.maps.Marker({
        position: coordinates,
        map: gmap,
        label: { text: text }
    });

    return marker;
}

var gmap = new google.maps.Map(document.getElementById('map'),
    {
        zoom: 4,
        center: { lat: 50.43333333, lng: 30.516667 }
    });

var marker = add_marker(gmap, { lat: 50.43333333, lng: 30.516667 }, 'UA');
// just hide it
marker.setVisible(false);
// or completely remove from a map
marker.setMap(null);

How to auto center map on markers

Suppose you have a bunch of stores in different countries scattered by cities. What you want is to guide user through all of them and auto-center map on interested country. There are at least two method:

  • Calculate centroid on list of points in specific country.
  • Use LatLngBounds class to extend the bounds of rectangle from list of points and then compute the center using getCenter method.

Live preview or view source code

auto_center
Moving through list of places

How to add custom data to marker

Just set the properties of Marker class to some predefined data. Let’s look into example: When someone click on marker – alert custom data info:

<script>
function add_marker(gmap, coordinates, text) {
  var marker = new google.maps.Marker({
    position: coordinates,
    map: gmap,
    label: { text: text }
  });

  return marker;
}

function init_map() {
  var places = {
    'UA': { lat: 50.43333333, lng: 30.516667 },
    'DE': { lat: 52.51666667, lng: 13.4 },
    'GL': { lat: 64.18333333, lng: -51.75 },
    'JP': { lat: 35.68333333, lng: 139.75 },
    'MG': { lat: -18.91666667, lng: 47.516667 }
  };

  var gmap = new google.maps.Map(document.getElementById('map'),
    {
      zoom: 2,
      center: { lat: -25.363, lng: 131.044 }
    });

  for (var country in places) {
    var marker = add_marker(gmap, places[country], country);
    marker.countryInfo = country;
    google.maps.event.addListener(marker, 'click', function () {
      alert('Clicked on ' + this.countryInfo + ' country');
    });
  }
}
</script>
<script src="https://maps.googleapis.com/maps/api/js?callback=init_map"></script>

How to get coordinates when moving marker

If you want to track position of marker when user is moving it, you need to listen on the following events:

  • dragstart: user started dragging marker.
  • drag: in process of dragging.
  • dragend: finished.

Here is the excerpt from source code to listen on the above events:

var marker = add_marker(gmap, places[country], country);
marker.countryInfo = country;
google.maps.event.addListener(marker, 'dragstart', function (event) {
    console.log(this.countryInfo + ': Start position: ', coords2text(this.getPosition()));
});
google.maps.event.addListener(marker, 'drag', function (event) {
    console.log(this.countryInfo + ': Current position: ', coords2text(this.getPosition()));
});
google.maps.event.addListener(marker, 'dragend', function (event) {
    console.log(this.countryInfo + ': Finished at: ', coords2text(this.getPosition()));
});

Live preview or view source code

How to adjust zoom level to include all visible markers on screen

There is no need to calculate distance between markers, trying to get scale ratio for zoom levels, etc., because there exists function fitBounds, which doing all the mentioned steps under the hood. So, in order to automatically adjust zoom level according to number of marker on the map, all is needed is to extend LatLngBounds to included all markers position and then call fitBounds function.

Live preview or view source code

How to resize marker image

When using custom icon in marker the default size of icon is the original size of image. In order to scale image to particular size the scaledSize property must be used.

function add_marker(gmap, coordinates, icon) {
    var marker = new google.maps.Marker({
        position: coordinates,
        map: gmap,
        icon: icon
    });

    return marker;
}

function init_map() {
    var places = {
        'UA': { lat: 50.43333333, lng: 30.516667 },
        'DE': { lat: 52.51666667, lng: 13.4 },
        'GL': { lat: 64.18333333, lng: -51.75 },
        'JP': { lat: 35.68333333, lng: 139.75 },
        'MG': { lat: -18.91666667, lng: 47.516667 }
    };

    var gmap = new google.maps.Map(document.getElementById('map'),
        {
            zoom: 2,
            center: { lat: -25.363, lng: 131.044 }
        });

    var size = 25;
    for (var country in places) {
        var icon = {
            url: "images/pin.png",
            scaledSize: new google.maps.Size(size, size),
        };
        add_marker(gmap, places[country], icon);
        size += 10;
    }
}

Live preview or view source code

How to add info window to marker

Google maps contains built-in class for showing info window over some map element, named InfoWindow. Using and styling info windows is pretty easy:

  • Content property – string with HTML elements that would be located in rectangular area in predefined position.
  • Open method which receive map and anchor as input in order to show info window.

Live preview or view source code

How to add marker using URL format

When you want to put marker on a Google map without writing code at all, you can use the following URL format:

http://maps.google.com/maps?&z=ZOOM_LEVEL&q=LATITUDE+LONGITUDE

Try this one: http://maps.google.com/maps?&z=9&q=35.68333333+139.75

How to use Font Awesome as marker icon

Fontawesome is a set of vector icons that be easily customized using CSS.  It contains nearly 675 icons divided by categories which makes it perfect candidate for custom marker icon.

In order to use Fontawesome as custom marker icon we need to switch from Marker class to RichMarker class. The latter designed for defining marker as arbitrary DOM element. Now, after switching to RichMarker library it’s very easy to define custom icon: just copy name of icon from Fontawesome website to the RichMarker content property.

function add_marker(gmap, coordinates, content) {
    var marker = new RichMarker({
        position: new google.maps.LatLng(coordinates),
        map: gmap,
        flat: true,
        anchor: RichMarkerPosition.MIDDLE,
        content: content
    });

    return marker;
}
// adding gear spinning animation as marker icon
add_marker(gmap, { lat: 48.857487, lng: 33.222656 },
   '<i class="fa fa-gear fa-spin fa-fw fa-2x" style="color:green"></i>');

Live preview or view source code

How to use SVG as marker icon

Marker class has a possibility to define custom marker icon as SVG path data, which is very powerful concept as you can mix vector/raster graphics in XML. There are predefined set of built-in symbol paths which can be customized using icon properties like: scale, strokeWeight, strokeColor, fillColor, etc.

In the linked demo, you can find basic usage of SVG icon. We are using predefined CIRCLE path and customized color, scale, fillColor, etc in order to show population by country. The larger population the bigger scale and color depth.

Live preview or view source code

Server-side Markers clustering using PostgreSQL + PostGIS and NodeJS

In this post I will discuss clustering of points on server side using combination of PostgreSQL and PostGIS extension. Then there will be presented NodeJS application, which simply pulls data from database using set of latitude and longitude describing interested area.

Why to cluster places?

Suppose you are working on a website which allows everyone to pay some fee to plant a tree in chosen location. For each tree there is a pin on a map with tree marker and the name of generous person who skipped mornings coffee for the sake of restoring forest around the world. Now, imagine for a moment when your website became popular and you have 10.000 of trees planted in some small area, now your map looks like this:

tree_pins
Ughhh, where on earth those trees located?

Wouldn’t it be better to gather trees into some kind of bubble, put number of trees on it, so the person could distinguish high density of trees and make decision if that’s a good place to plant another one?

clustered_tree_pins
yes, that’s much better

Installing and Setup Prerequisite Software

Here is the list of what need to be installed beforehand:

  • PostgreSQL
  • PostGIS extension
  • NodeJS

When the installation of PostgreSQL + PostGIS is done, we need to create new database and enable PostGIS extension. Now fire up psql in terminal and type the following commands:


# start psql with default postgres user

psql -U postgres

# create database where our places will be located

CREATE DATABASE trees;

# work on trees database

\connect trees;

# now enable PostGIS extension, so we can use all spatial functions, data types, etc

CREATE EXTENSION postgis;

Clustering algorithm

In our case there will be used pretty simple clustering approach:

  • Get any point that not part of existing cluster.
  • Find all points in predefined radius that not part of existing cluster.
  • Mark found points that they belong to particular cluster.
  • Repeat while there will be no free points without cluster.

PostgreSQL + PostGIS implementation

Before start writing SQL queries we need to understand the basic idea of clustering points. There will be implemented dummy approach of finding points in specific radius and this radius will be defined for each zoom level. So, when on zoom level two: the radius will be 100 km and then while zooming in: the radius will be approximately divided by two. This is slow solution as for each zoom level we need to gather all points using different radius, the more optimized solution will be clustering on largest zoom level and then using clustered points as input to clustering for upper zoom level and so on.

Now we want to create table named places, with place latitude and longitude and cluster id for each zoom level. Cluster id is additional column to identify cluster this point belongs to. For each zoom level the additional table will be created with the name clusters_zoom


-- create table places and columns named cluster<n>
-- where n - is the cluster for specific zoom level.
-- In our case we want to cluster points in zoom level range [2..16]

CREATE TABLE places(id SERIAL PRIMARY KEY,
                    place GEOGRAPHY,
                    cluster2 INTEGER,
                    cluster3 INTEGER,
                    cluster4 INTEGER,
                    cluster5 INTEGER,
                    cluster6 INTEGER,
                    cluster7 INTEGER,
                    cluster8 INTEGER,
                    cluster9 INTEGER,
                    cluster10 INTEGER,
                    cluster11 INTEGER,
                    cluster12 INTEGER,
                    cluster13 INTEGER,
                    cluster14 INTEGER,
                    cluster15 INTEGER,
                    cluster16 INTEGER,
                    dummy INTEGER);
-- create table for each zoom level with the following fields:
-- cluster: id of this cluster, this will be stored in cluster<n> column
-- places table.
-- pt_count: number of points in this cluster
-- centroid: average position of all the points in cluster
CREATE TABLE clusters_zoom_2(cluster SERIAL PRIMARY KEY,
                             pt_count INTEGER,
                             centroid GEOMETRY);

First of all we need sample data with places we want to cluster. The easiest way is to generate random points in predefined bounding box. In our example the bounding box will be located in San Francisco:

sf_bbox
Sample data generated in bounding box.
-- generate 10.000 random points in bounding box with coordinates (longitude, latitude):
-- south-west: (-122.51478829956056, 37.686456995336954)
-- north-east: (-122.3220125732422, 37.79505521136725)
DO
$do$
BEGIN
 FOR i IN 1..10000 LOOP
  INSERT INTO places(place, dummy) VALUES (ST_MakePoint(
    random()*(-122.3220125732422 - -122.51478829956056) + -122.51478829956056,
    random()*(37.79505521136725 - 37.686456995336954) + 37.686456995336954
    ), i);
 END LOOP;
END
$do$;
-- create index on places table
CREATE INDEX places_index ON places USING GIST (place);

Here is the main SQL function which is executed for each zoom level and gather points into clusters with the predefined radius. In this example the function is hard-coded for zoom level two and will be automated to execute for each zoom level in NodeJS module.

The make_cluster function is pretty straightforward:

All points are gathered using ST_DWithin function, which returns all geometries within the specified distance of one another. Pay attention to the fact that ST_DWithin takes as input geometry or geography type as first parameter and distance as the last one. In our case we use geography as we need to measure distance in meters.

After new cluster is created then the clusters_zoom and places tables are updated with the new cluster id.

CREATE FUNCTION make_cluster2() RETURNS INTEGER AS
$$
    DECLARE start_place GEOGRAPHY;
    DECLARE cluster_id INTEGER;
    DECLARE ids INTEGER[];
      BEGIN
        SELECT place INTO start_place FROM places WHERE cluster2 IS NULL limit 1;
        IF start_place is NULL THEN
            RETURN -1;
        END IF;
        SELECT array_agg(id) INTO ids FROM places WHERE cluster2 is NULL AND ST_DWithin(start_place, place, 100000);
        INSERT INTO clusters_zoom_2(pt_count, centroid)
         SELECT count(place), ST_Centroid(ST_Union(place::geometry)) FROM places, unnest(ids) as pid
        WHERE id = pid
        RETURNING cluster INTO cluster_id;
        UPDATE places SET cluster2 = cluster_id FROM unnest(ids) as pid WHERE id = pid;
        RETURN cluster_id;
      END;
$$  LANGUAGE plpgsql;

Now, we are running this function to fill clusters_zoom table, while there exists points without cluster assigned to it:

DO
$do$
DECLARE cluster_id INTEGER;
BEGIN
    SELECT 0 INTO cluster_id;
    WHILE cluster_id != -1
    LOOP
     SELECT make_cluster2() INTO cluster_id;
    END LOOP;
END
$do$;

NodeJS implementation

Let’s assume that we have a table with all trees latitude, longitude and some additional data specified by user. What we want is to start clustering process in background as it takes some time to finish and then notify user that all data was processed successfully.

In order to do this we can create separate module called clusterapp.js which will be running as background process with database connection string as input parameter.

Before running clusterapp.js the following Node modules need to be installed:

The source code is pretty simple and automate SQL queries from the steps above:

var fs = require('fs');
var pgp = require('pg-promise')();
var db;

// predefined clustering radius for each zoom level (in meters)
var zoom_level_radius =
{
    16: 100,
    15: 200,
    14: 500,
    13: 1000,
    12: 2000,
    11: 3000,
    10: 4000,
    9: 7000,
    8: 15000,
    7: 25000,
    6: 50000,
    5: 100000,
    4: 200000,
    3: 400000,
    2: 700000
};

process.on('config', (config, callback) => {
    console.log('clusterapp: Configuration: ', config);
    // connecting to PostgreSQL instance
    // example: postgres://dbname:dbpassword@localhost:5432/places
    db = pgp(config['connection_string']);
    callback();
    start_clustering();
});

String.prototype.format = function()
{
    var formatted = this;
    for (var i = 0; i < arguments.length; i++) {
var regexp = new RegExp('\\{'+i+'\\}', 'gi');
formatted = formatted.replace(regexp, arguments[i]);
}
return formatted;
};
// create cluster table for each zoom level
function create_cluster_zooom_tables() {
db.task(t => {
        return t.batch([
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [2]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [3]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [4]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [5]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [6]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [7]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [8]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [9]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [10]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [11]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [12]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [13]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                    " centroid GEOMETRY, classify INTEGER);", [14]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                      " centroid GEOMETRY, classify INTEGER);", [15]),
                t.none("CREATE TABLE IF NOT EXISTS clusters_zoom_$1(cluster SERIAL PRIMARY KEY, pt_count INTEGER," +
                       " centroid GEOMETRY, classify INTEGER);", [16])
        ]);
    }).then(data => {
        console.log("create_cluster_zooom_tables: cluster tables created: " + data);
    }).catch(error => {
        console.log("create_cluster_zooom_tables:ERROR: " + error);
        finish_clustering({'ERROR': 'Creating tables failed: ' + error});
    });
}

// create make_cluster function for each zoom level
function create_cluster_functions()
{
    var cluster_func_query =
        "CREATE OR REPLACE FUNCTION make_cluster{0}() RETURNS INTEGER AS\n" +
        "$$\n" +
        "DECLARE start_place GEOGRAPHY;\n" +
        "DECLARE cluster_id INTEGER;\n" +
        "DECLARE ids INTEGER[];\n" +
         "BEGIN\n" +
            "SELECT place INTO start_place FROM places WHERE cluster{0} IS NULL limit 1;\n" +
            "IF start_place is NULL THEN\n" +
                "RETURN -1;\n" +
            "END IF;\n" +
            "SELECT array_agg(id) INTO ids FROM places WHERE cluster{0} is NULL AND ST_DWithin(start_place, place, {1});" +
            "INSERT INTO clusters_zoom_{0}(pt_count, centroid)\n" +
            "SELECT count(place), ST_Centroid(ST_Union(place::geometry)) FROM places, unnest(ids) as pid\n" +
            "WHERE id = pid\n" +
            "RETURNING cluster INTO cluster_id;\n" +
            "UPDATE places SET cluster{0} = cluster_id FROM unnest(ids) as pid WHERE id = pid;\n" +
            "RETURN cluster_id;\n" +
        "END;\n" +
        "$$  LANGUAGE plpgsql";

    for (var zoom_level in zoom_level_radius)
    {
        var query_create_func = cluster_func_query.format(zoom_level, zoom_level_radius[zoom_level]);
        db.none(query_create_func)
          .then(result => {
              console.log("create_cluster_functions: function created successfully");
          })
          .catch(error => {
              console.log('create_cluster_functions: Creating clustring function failed due: ' + error);
              finish_clustering({'ERROR': 'Creating functions failed: ' + error});
          });
    }
}

// creating cluster for each zoom level
function make_clusters()
{
    var query =
            "DO\n" +
            "$do$\n" +
            "DECLARE cluster_id INTEGER;\n" +
            "BEGIN\n" +
                "SELECT 0 INTO cluster_id;\n" +
                "WHILE cluster_id != -1\n" +
                "LOOP\n" +
                "SELECT make_cluster$1() INTO cluster_id;\n" +
                "END LOOP;\n" +
            "END\n" +
            "$do$;";

    db.task(t => {
        return t.batch([
                t.none(query, [2]),
                t.none(query, [3]),
                t.none(query, [4]),
                t.none(query, [5]),
                t.none(query, [6]),
                t.none(query, [7]),
                t.none(query, [8]),
                t.none(query, [9]),
                t.none(query, [10]),
                t.none(query, [11]),
                t.none(query, [12]),
                t.none(query, [13]),
                t.none(query, [14]),
                t.none(query, [15]),
                t.none(query, [16])
        ]);
    }).then(data => {
        console.log("make_clusters: cluster created: " + data);
        finish_clustering({'OK': ''});
    }).catch(error => {
        console.log("make_clusters:ERROR: " + error);
        finish_clustering({'ERROR': 'Creating clusters failed: ' + error});
    });
}

function start_clustering()
{
    create_cluster_zooom_tables();
    create_cluster_functions();
    make_clusters();
}

function finish_clustering(status)
{
    pgp.end();
    process.send({'status': status});
}

After the clustering process is done we need to think about how to use it in our application. So, we have backend as PostgreSQL database with a bunch of tables with clustered points and we need API layer in order to retrieve clusters for selected area and specific zoom level.

In order to gather all points in specific area we can use PostGIS && bounding box intersect operator on centroid from clustered points table and actual bounding box of interested area created using ST_MakeEnvelope function.

Here is the full query:

-- get all clusters for second zoom level in area near San Francisco
SELECT cluster,
       pt_count,
       centroid
FROM  clusters_zoom_2
WHERE centroid && ST_MakeEnvelope(-122.51478829956056, 37.686456995336954,
                                  -122.3220125732422, 37.79505521136725, 4326);

In the NodeJS implementation we create api.js module with two endpoints:

  • clusters: Getting all clusters at zoom level and bounding box passed as parameter.
  • places: Getting original, non-clustered data at specific zoom level and bounding box.
var express = require('express');
var router = express.Router();
var pgp = require('pg-promise')();
var backgrounder = require('backgrounder');
var GeoJSON = require('geojson');
// database connection parametres
var db = pgp('postgres://dbname:dbpassword@localhost:5432/places');

// optimization: define buffer radius for each zoom level
// when retrieving data for specific bounding box
var zoom_level_buffer_radius =
{
    16: 1000,
    15: 1000,
    14: 1000,
    13: 1000,
    12: 1000,
    11: 1500,
    10: 2000,
    9: 3000,
    8: 5000,
    7: 12000,
    6: 25000,
    5: 50000,
    4: 100000,
    3: 100000,
    2: 100000
};

// zoom: zoom level
// ne_lat, ne_lng: north east latitude and longitude
// sw_lat, sw_lng: south west latitude and longitude
// http://localhost:3000/api/clusters?id=&zoom=2&ne_lat=-21.32940556631426&ne_lng=146.66655859374998&sw_lat=-29.266381110600395&sw_lng=115.42144140624998
router.get('/clusters', function(req, res, next) {
    let query_get_clusters = "SELECT cluster, pt_count, ST_X(centroid) as lng, ST_Y(centroid) as lat FROM \n" +
                                    "clusters_zoom_$1 WHERE \n" +
                                    "centroid && ST_MakeEnvelope($2, $3, $4, $5, 4326);";
    let query_get_buffer = "SELECT ST_XMin(bu::geometry) as sw_lng,\n" +
                                   "ST_YMin(bu::geometry) as sw_lat, ST_XMax(bu::geometry) as ne_lng,\n" +
                                   "ST_YMax(bu::geometry) as ne_lat FROM\n" +
                                   "ST_Buffer(ST_GeographyFromText(ST_AsEWKT(ST_MakeEnvelope($1, $2, $3, $4, 4326))), 1000) as bu;"

    db.manyOrNone(query_get_buffer, [parseFloat(req.query['sw_lng']),
                                     parseFloat(req.query['sw_lat']),
                                     parseFloat(req.query['ne_lng']),
                                     parseFloat(req.query['ne_lat']),
                                     zoom_level_buffer_radius[parseInt(req.query['zoom'])]])
                                     .then(buffered_box_data => {
                                        db_connection.manyOrNone(query_get_clusters, [parseInt(req.query['zoom']),
                                                                                               buffered_box_data[0].sw_lng,
                                                                                               buffered_box_data[0].sw_lat,
                                                                                               buffered_box_data[0].ne_lng,
                                                                                               buffered_box_data[0].ne_lat])
                                                    .then(clusters_data => {
                                                        var result = GeoJSON.parse(clusters_data, {Point: ['lat', 'lng']});
                                                        result['buffered_bbox'] = [buffered_box_data[0].sw_lng,
                                                                                   buffered_box_data[0].sw_lat,
                                                                                   buffered_box_data[0].ne_lng,
                                                                                   buffered_box_data[0].ne_lat];
                                                        res.json(result);
                                                    }).catch(error => {
                                                        console.log(error);
                                                        res.json({'ERROR': 'Invalid request'});
                                                    });
                                                }).catch(error => {
                                                    console.log(error);
                                                    res.json({'ERROR': 'Invalid request'});
                                                });
});

// ne_lat, ne_lng: north east latitude and longitude
// sw_lat, sw_lng: south west latitude and longitude
// http://localhost:3000/api/places?id=&ne_lat=-21.32940556631426&ne_lng=146.66655859374998&sw_lat=-29.266381110600395&sw_lng=115.42144140624998
router.get('/places', function(req, res, next) {
    let query_string = "SELECT id, ST_X(place::geometry) as lng, ST_Y(place::geometry) as lat \n" +
                       "FROM places WHERE place && ST_MakeEnvelope($1, $2, $3, $4, 4326);";
    db.manyOrNone(query_string, [parseFloat(req.query['sw_lng']),
                                 parseFloat(req.query['sw_lat']),
                                 parseFloat(req.query['ne_lng']),
                                 parseFloat(req.query['ne_lat'])])
                                 .then(data => {
                                            res.json(GeoJSON.parse(data, {Point: ['lat', 'lng']}));
                                        }).catch(error => {
                                            res.json({'ERROR': 'Invalid request'});
                                        });
});

module.exports = router;

Now, you can run api.js module and start testing query in your browser. The response result for each query will be GeoJSON with data that was pulled from database.

How to use in real application?

It doesn’t matter what map provider you are using as you have server-side clustering with API layer on top of it.

Here is basic example using google maps, which is listening on bounds_changed event and getting all clusters inside current bounding box:

</pre>
<pre><!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
<style>
       #map {
        height: 800px;
        width: 80%;
       }
    </style>

  </head>
  <body>
<div id="map"></div>
<script>
        var gmap;

        function show_clusters(zoom, ne_lat, ne_lng, sw_lat, sw_lng, bounds) {
          http_request = new XMLHttpRequest();
          http_request.onreadystatechange = function() {
            if (this.readyState === XMLHttpRequest.DONE) {
              if (this.status === 200) {
                var response = JSON.parse(this.responseText);
                if (response.hasOwnProperty('ERROR')) {
                  console.log('ERROR: ', response['ERROR']);
                } else {
                  response['features'].forEach(feature => {
                    var pt = {lng: parseFloat(feature['geometry']['coordinates'][0]),
                              lat: parseFloat(feature['geometry']['coordinates'][1])};
                    console.log('cluster: ', pt, ' points: ', feature['properties']['pt_count'].toString());
                  });
                }
              } else {
                console.log('show_clusters: ERROR status: ', this.status);
              }
            }
          };
          var params = 'id=' + id + '&' +
                       'zoom=' + zoom + '&' +
                       'ne_lat=' + ne_lat + '&' +
                       'ne_lng=' + ne_lng + '&' +
                       'sw_lat=' + sw_lat + '&' +
                       'sw_lng=' + sw_lng;

          var query = 'http://localhost:3000/api/clusters?' + params;
          http_request.open('GET', query);
          http_request.send();
        }

        function bounds_changed() {
          var bounds = gmap.getBounds();
          var SW = bounds.getSouthWest();
          var NE = bounds.getNorthEast();

          var ne_lat = NE.lat();
          var ne_lng = NE.lng();
          var sw_lat = SW.lat();
          var sw_lng = SW.lng();

          show_clusters(zoom, ne_lat, ne_lng, sw_lat, sw_lng, bounds);
        }

        function init_map() {
          gmap = new google.maps.Map(document.getElementById('map'),
            {
              zoom: 1,
              center: { lat: -25.363, lng: 131.044 }
            });

          google.maps.event.addListener(gmap, 'bounds_changed', bounds_changed);
        }

    </script>
    <script src="https://maps.googleapis.com/maps/api/js?callback=init_map"></script>
  </body>
</html>

Optimization

As you probably noticed in api.js module, when getting clusters from PostgreSQL there used ST_Buffer function, which extend bounding box on some predefined distance. This is useful when you are trying to get clusters on bounds changed event. It’s better to make one query with extended visible area and then use cached result when user is panning map than flood your server with query for every bounds change.

Another thing that can speed-up clustering process it to perform hierarchical clustering. Instead of running the same clustering function on all places in your database and just changing the radius basing on zoom level, we can run this function on previously clustered data. So, for example the input points for clustering function for ninth zoom level would be clustered data from tenth zoom level and so on.

OpenLayers: Align label with Line Feature

In this short post we are going to discuss how to display label on the LineString. This can be useful if you want to add some information along the line, for example start and destination city:

ol_rotating_text_slope_logo
Label along the LineString geometry

In our case the label would be OpenLayer’s Text style for vector feature. It contains property named: rotation, which contains rotation in radians (positive rotation clockwise).

In order to calculate rotation value we need to do bearing to angle conversion (for more information on math behind this: Calculate distance, bearing and more between Latitude/Longitude points):

function radians(n) {
    return n * (Math.PI / 180);
}

function degrees(n) {
    return n * (180 / Math.PI);
}

function bearing(start_lat, start_long, end_lat, end_long) {
    start_lat = radians(start_lat);
    start_long = radians(start_long);
    end_lat = radians(end_lat);
    end_long = radians(end_long);

    var dlong = end_long - start_long;

    var dphi = Math.log(Math.tan(end_lat / 2.0 + Math.PI / 4.0) /
            Math.tan(start_lat / 2.0 + Math.PI / 4.0));
    if (Math.abs(dlong) > Math.PI) {
        if (dlong > 0.0)
            dlong = -(2.0 * Math.PI - dlong);
        else
            dlong = (2.0 * Math.PI + dlong);
    }

    return (degrees(Math.atan2(dlong, dphi)) + 360.0) % 360.0;
}

function bearingToRadians(br) {
    return radians((450 - br) % 360);
}

function rotation(pt0, pt1, br) {
    var rotate = pt0[0] > pt1[0] ? Math.PI : 0;
    return bearingToRadians(br) + rotate
}

Here is the source code which add label to Line feature and display text on it, you can view it on github:

function radians(n) {
    return n * (Math.PI / 180);
}

function degrees(n) {
    return n * (180 / Math.PI);
}

function bearing(start_lat, start_long, end_lat, end_long) {
    start_lat = radians(start_lat);
    start_long = radians(start_long);
    end_lat = radians(end_lat);
    end_long = radians(end_long);

    var dlong = end_long - start_long;

    var dphi = Math.log(Math.tan(end_lat / 2.0 + Math.PI / 4.0) /
            Math.tan(start_lat / 2.0 + Math.PI / 4.0));
    if (Math.abs(dlong) > Math.PI) {
        if (dlong > 0.0)
            dlong = -(2.0 * Math.PI - dlong);
        else
            dlong = (2.0 * Math.PI + dlong);
    }

    return (degrees(Math.atan2(dlong, dphi)) + 360.0) % 360.0;
}

function bearingToRadians(br) {
    return radians((450 - br) % 360);
}

function rotation(pt0, pt1, br) {
    var rotate = pt0[0] > pt1[0] ? Math.PI : 0;
    return bearingToRadians(br) + rotate
}

var wsg84_pt1 = [-74.0059, 40.7127];
var wsg84_pt2 = [-75.6972, 45.4215];
// convert to default projection EPSG:3857 Web Mercarator
var pt1 = ol.proj.fromLonLat(wsg84_pt1);
var pt2 = ol.proj.fromLonLat(wsg84_pt2);

var map = new ol.Map({
    layers: [
        new ol.layer.Tile({
            source: new ol.source.OSM()
        })
    ],
    target: 'map',
    view: new ol.View({
        center: pt1,
        zoom: 6
    })
});
var FEATURE_LINE = 0;
var FEATURE_PT1 = 1;
var FEATURE_PT2 = 2;
var line = new ol.Feature({
    geometry: new ol.geom.LineString([pt1, pt2])
});
var pt1_feature = new ol.Feature({
    geometry: new ol.geom.Point(pt1)
});
var pt2_feature = new ol.Feature({
    geometry: new ol.geom.Point(pt2)
});
line.feature_type = FEATURE_LINE;
pt1_feature.feature_type = FEATURE_PT1;
pt2_feature.feature_type = FEATURE_PT2;

var feature_style = function (feature) {
    if (FEATURE_LINE == feature.feature_type) {
        return new ol.style.Style({
            stroke: new ol.style.Stroke({
                color: 'rgba(0, 204, 255, 0.6)',
                width: 10
            }),
            text: new ol.style.Text({
                rotation: -rotation(pt1, pt2, bearing(wsg84_pt1[1], wsg84_pt1[0], wsg84_pt2[1], wsg84_pt2[0])),
                text: 'New York to Ottawa',
                font: '17px sans-serif',
                fill: new ol.style.Fill({color: 'white'}),
                stroke: new ol.style.Stroke({color: 'black', width: 2})
            })
        });
    } else {
        var fill_color;
        switch (feature.feature_type) {
            case FEATURE_PT1:
                fill_color = new ol.style.Fill({color: 'rgba(191, 63, 191, 0.6)'});
                break;
            case FEATURE_PT2:
                fill_color = new ol.style.Fill({color: 'rgba(63, 191, 63, 0.6)'});
                break;
        }
        return new ol.style.Style({
            image: new ol.style.Circle({
                fill: fill_color,
                radius: 6,
                stroke: new ol.style.Stroke({color: 'black', width: 1.25})
            })
        });
    }
};
var vector_source = new ol.source.Vector({});
vector_source.addFeatures([pt1_feature, pt2_feature, line]);
var vector_layer = new ol.layer.Vector({
    source: vector_source,
    map: map,
    style: feature_style
});