Live data from Hacker News

PostgreSQL is the worlds’ best database

2ndquadrant.com

311–320 of 365 posts

Re: PostgreSQL is the worlds’ best database

#311

Earlier quoted context omitted.

So you have more complicated SQL then, but it doesn't make a difference between DBs selecting from a 100M table with indexes. If it ends up being a full table scan then PG only recently got query compilation and parallel execution so at best it would be even with, but not better than, MSSQL/Oracle that can do vectorized executions on columnstores.

You need different index structures. Yes, it matters. Hard to believe that spatial queries over geometries more complex than points will meaningfully benefit from "vectorized executions on columnstores".

Do you have an example query? If you're not using geospatial features, then what indexes does PG offer that the others can't match (either directly with indexes or with better scan/query performance)?

Re: PostgreSQL is the worlds’ best database

#312
post #244

I get the Postgres love but as someone who’s been doing mysql for a long time I found postgres really hard to use. Like even setting it up on Mac was a pain in the ass. How do I create a new db? Why is pgcli not as friendly as mysqlcli? After 2 weeks of head bashing and lots of stack overflow reading I gave up and went back to good ol MySQL. For ease of use for a new comer, mysql seems to have nicer tooling and out o…

> setting it up on Mac was a pain in the ass

What was your problem? All you need to call is:

docker run --rm --name pg-docker -e POSTGRES_PASSWORD=docker -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres:latest

...and you are good to go. In depth guide: https://hackernoon.com/dont-install-postgres-docker-pull-pos...

Re: PostgreSQL is the worlds’ best database

#313
post #56

Having migrated a postgres database to dynamodb, I would say that I'm never going to use postgres again. The problem with all SQL databases is that they are too easy to query and use. You add all kinds of select queries, joins and foreign keys and when traffic hits scramble to make it scale. NoSQL is hard to design but you can atleast be sure that once traffic hits, you don't have to redesign the schema to make it sc…

You don't have joins in NoSQL so you have to keep data denormalized. That means data isn't consistent and you can have an will have (at a high enough data volume) errors if you aren't extremely cautious on how you update and add new data. That might matter or not in a particular business case.

I really hope the GP doesn't work on anything handling financial transactions.

Re: PostgreSQL is the worlds’ best database

#314
That performance carries the day today is a huge improvement since 1999. Back then, there was also only one question that mattered, but the question was, "Is it Oracle?", just as the only question about network equipment was "Is it Cisco?".

It is hard to explain, at this remove, what horrible perversions were performed in order to be able to say "yes" to such questions, or to avoid need to answer them.

One example was a router that, at opposite ends of a long-haul network, managed data flow that was utterly insensitive to the magnitudes of latency or drop rate. They sold hardly any. It turned out the algorithm could be run in user space over UDP, and that became a roaring business, for a different company, because you could deploy without getting network IT involved. Mostly IT didn't even want bribes. They just didn't want anything around that was unfamiliar.

Database products had to pretend to be an add-on to Oracle, and invisibly back up to, and restore from, an Oracle instance, because Oracle DBAs only ever wanted to back up or restore an Oracle instance. The Oracle part typically did absolutely nothing else, but Oracle collected their $500k (yes, really) regardless.

Re: PostgreSQL is the worlds’ best database

#315
post #302
post #83

Earlier quoted context omitted.

You could design a SQL database to be denormalized from the start, but then you are losing many of the advantages of a SQL database. I never said that schemas aren't necessary, just that using a strict NoSQL database forces you to think about scaling constraints early. This avoids(atleast partially) a schema redesign later.

Ah yes! Optimize early, that's what my teachers always told me to do. /s

Code is easy to change. Data schemata much less so, particularly in traditional SQL databases.

Re: PostgreSQL is the worlds’ best database

#316
> So, you want NoSQL, Riak, REACT, Redis, Mongo, etc.? PostgreSQL does all that. Admittedly not with all the bells and whistles of all of the original products. For example, PostgreSQL doesn’t create new shards for you for any of those. That’s still a manual process.

Isn't this a fairly big issue? I would think the convenience of automated scaling is a primary reason to use these other tools.

Re: PostgreSQL is the worlds’ best database

#317
post #82

Earlier quoted context omitted.

Oracle has not managed the shift to cloud so if you look at where the puck is going, it's moving away from Oracle.

Oracle has their own cloud platform and a great database to go along with it, if anything history has shown us that Oracle is able to adapt their database offerings to cater to current enterprise needs and I think this time will be no different.

The database is not the cloud platform. There is a lot of gravity there but it's not enough.

Re: PostgreSQL is the worlds’ best database

#318
post #82

Earlier quoted context omitted.

Oracle has not managed the shift to cloud so if you look at where the puck is going, it's moving away from Oracle.

The largest enterprise SaaS application in the world runs on Oracle.

Is that application built and sold by Oracle as well?

Re: PostgreSQL is the worlds’ best database

#319

Earlier quoted context omitted.

Good list. Also true indexed organized table aka real clustered indexes. Oh and real cross connection query plan caching, prepared statements are only for the connection and must be explicitly used. No need to use prepared statements in MSSQL since the 90's

Another thing I'd add is a quasi-sorted uuid generator built-in. Real clustered indexes need approximately sorted UUIDs - which could be version 1 UUIDs with an artificial, per-database MAC address, or they could be something more esoteric such as Twitter's snowflake ID generator. Using UUIDs for PKs is fine and dandy but clustered indexes and type 4 UUIDs do not play well. Many MS SQL users discover this the hard wa…

That's what NEWSEQUENTIALID() is for in MS SQL.

Re: PostgreSQL is the worlds’ best database

#320

For most of the projects where the DB really mattered, throughout my 10+ freelancer carrier, it came down to one thing that client really cared about. Performance. Nothing else mattered, not license price, not whistles and bells, not hype. My clients wanted to have data in front of their eyes the same second when they clicked the button. And when you have a table with 100 million rows in it, and an application is not…

Mysql can do this just fine with a geospatial index, or, for that matter, with a geohash index[1] (though in the case of a geohash index, you will need a somewhat more complex query). In fact, any database at all that can index on a string (i.e. all of them) can do efficient location-based search.

Honestly, if it's taking your database more than a second to pull 1000 of 100M rows on a simple query, that means you need to figure out what's wrong with your indexes, not that your choice of database vendor is bad. It is true that postgres is a very nice database, but for basic tasks, honestly any database will be fine if you know how to use it effectively.

Demo in mysql:

    -- create a helper table for inserting our 100M points
    create table nums (
      id bigint(20) unsigned not null
    );
    insert into nums (id)
    values (0), (1), (2), (3), (4), (5), (6), (7), (8), (9);

    -- the table in which we will store our 100M points
    create table points (
      id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
      latitude float NOT NULL,
      longitude float NOT NULL,
      latlng POINT NOT NULL,
      primary key (id),
      spatial key points_latlng_index (latlng)
    );

    -- insert the 100M points with latitude between 32 and 49
    -- and longitude between -120 and -75, which is roughly a
    -- rectangle covering the continental US, placing them
    -- randomly. Be patient, as it takes a while to create
    -- 100M rows.
    insert into points (latitude, longitude, latlng)
    select
      ll.latitude,
      ll.longitude,
      ST_GeomFromText(concat('POINT(',
        ll.latitude, ' ', ll.longitude,
      ')')) as latlng
    from (
      select
        32 + 17 * conv(left(sha1(concat('latitude', ns.n)), 8), 16, 10) / pow(2, 32) as latitude,
        -120 + 45 * conv(left(sha1(concat('longitude', ns.n)), 8), 16, 10) / pow(2, 32) as longitude
      from (
        select
          a.id * pow(10, 0) + b.id * pow(10, 1)
            + c.id * pow(10, 3) + d.id * pow(10, 4)
            + e.id * pow(10, 5) + f.id * pow(10, 6)
            + g.id * pow(10, 7) + h.id * pow(10, 8)
            as n
        from nums a join nums b join nums c join nums d
        join nums e join nums f
        join nums g join nums h
      ) as ns
    ) as ll;

    -- Generate a roughly circular boundary polygon 7km around
    -- the center of Las Vegas
    set @earth_radius_meters := 6371000,
        @lv_lat := 33.17,
        @lv_lng := -115.14,
        @search_radius_meters := 7000;
    set @boundary_polygon := (select
      ST_GeomFromText(concat('POLYGON((', group_concat(
        concat(boundary_lat, ' ', boundary_lng)
        order by id
        separator ', '), '))')
      ) as boundary_geom
    from (
      select
        nums.id,
        @lv_lat + @search_radius_meters * cos(nums.id * 2 * pi() / 9)
          / (@earth_radius_meters * 2 * pi() / 360) as boundary_lat,
        @lv_lng + @search_radius_meters * cos(@lv_lat * 2 * pi() / 360) * sin(nums.id * 2 * pi() / 9)
          / (@earth_radius_meters * 2 * pi() / 360) as boundary_lng
      from nums
    ) t);


    explain select count(*) from points where ST_Contains(@boundary_polygon, latlng)\G
    *************************** 1. row ***************************
               id: 1
      select_type: SIMPLE
            table: points
       partitions: NULL
             type: range
    possible_keys: points_latlng_index
              key: points_latlng_index
          key_len: 34
              ref: NULL
             rows: 987
         filtered: 100.00
            Extra: Using where
    1 row in set, 1 warning (0.01 sec)

    mysql> select count(*) from points where ST_Contains(@boundary_polygon, latlng)\G
    *************************** 1. row ***************************
    count(*): 1228
    1 row in set (0.01 sec)
Post reply on HN