My DS3231 test setup

I wanted to test several DS3231 (M and non-M variants) boards for drift, so I mounted eleven of them (including one known-genuine DS3231M, the leftmost one on the front row, with a green bodge wire) to a breadboard, connected a regulated power supply (AMS1117) at 3.3V to both power rails, and made sure they all worked.

Eleven DS3231 (including 3 M variants) on a breadboard for testing.

Yup, they all work. The boards have either orange or red LEDs, so they emit a pleasing glow at night that prevents me from crashing into things in my office at home.

Why use 3.3V? One, it makes interfacing with the 3.3V I2C pins on a Raspberry Pi easy since I don’t need a level-shifter, and two, it minimizes drift in the event that I need to disconnect the power and have the clocks run on their coin cell backups. The CR2032 batteries have a nominal voltage of 3.0V, but all currently measure 3.3V (they’re brand-new, Energizer-brand cells from Digi-Key). The DS3231 datasheet says the drift can change by up to 1ppm/volt, so I want to minimize the voltage difference between the normal power supply and the coin cells.

To ease the comparison of drift, I want to ensure all the clocks start counting at the same moment. I could set them all one at a time, but this is complicated because (a) I don’t have an I2C multiplexer chip, and (b) setting them sequentially means they’re not all set at the same moment. It probably wouldn’t matter much in the long run, but it would make me happy to set them at the same time.

The DS3231 modules all have the I2C address of 0x68, and it cannot be changed. Normally, you cannot have multiple chips with the same address on the same I2C bus, as they’ll talk all over each other and the resulting signals will be garbage.

Fortunately, we don’t need the DS3231s to talk; they need only listen to the master and make the appropriate ACK/NAK signals as needed. They should all send the same ACK/NAK signals at the same time so, in theory, there shouldn’t be a problem.

Next, we need to worry about bus current. Each module has a 4.7k ohm pull-up resistor for the I2C bus. With eleven modules, that means the effective pull-up resistance is ~430 ohms. At 3.3V working voltage, a device would need to sink nearly 8mA to correctly signal a logic low. The Raspberry Pi I have can sink 16mA per GPIO pin, so that’s fine. The DS3231 datasheet says the IOL is 3mA, though I spoke with an engineer at Maxim Semiconductor and they said the absolute maximum current the process used on the chips is 10mA. 8mA is close to that limit, but the current would hopefully be spread across many devices and would only be for a few microseconds in total, so it should be fine.

I was satisfied I wasn’t going to blow anything up (and if I did, replacements are cheap), so I connected all eleven modules in parallel to the same I2C bus and commanded them to set their date and time to an arbitrary date in the past. If this was successful, I could send a command to read the time and, if all the modules had the same time, it would come through without an error. If things didn’t work, garbage would come in and I’d have to check them individually for the correct time. One read to all of the devices simultaneously, and I had valid data for that arbitrary time and date. Excellent. It worked!

Using the Raspberry Pi synchronized to a local NTP server (another Raspberry Pi running NTP with a GPS reference clock) within less than a millisecond, I send the command to set the date and time on all the modules to the current time on Friday 8 Sep 11:18:16 UTC 2017 (unix time: 1504869496). Reading the date and time from all the modules confirms they all have the correct date and time with no errors.

Now I’ll let them run for a while to see how they drift. A few have hand-tuned aging registers, so they should hopefully drift less than the others, while others use the default aging register of 0.

 

It’s the law: Certificate Authority websites must suck.

I’m pretty sure that it’s some sort of universal law that all Certificate Authority websites must be filled with obfuscating marketing-ese wording, links to “white papers”, contradictory and uninformative text, and content generally tailored for manager-types.

Honestly, I don’t know why they do this: TLS certificates are essentially always handled by technical staff — not management — at companies. Smaller organizations typically leave the administration of TLS certs to their commercial web hosts (again, technical staff). Individual site operators either know how to handle certs or don’t, but for those who don’t the marketing fluff on a CA website isn’t likely to help at all.

There may be some very specific reason why a particular CA is required, such as needing to support particular software or devices that only include a limited selection of roots, and while these reasons may be decided by managers and executives, the actual deployment is done by technical staff. The CA websites should really be tailored for technical people, not managers.

In addition to the typical manager-speak found on CA websites, the amount of confusing information is shocking. Some of it is merely misleading (e.g. implying that a particular certificate enables 128/256-bit symmetric ciphers rather than merely vouching for the identity of the server; the supported symmetric ciphers are set in the server configuration independently of the certificate and are negotiated with the client), while others are outright deceptive: Symantec/Thawte go so far as to claim that Server-Gated Cryptography is still relevant in this day and age (hint: it isn’t). In addition to being absurdly insecure and out of date, 16+ year old “export-grade” browsers that require SGC for strong cryptography are likely completely incapable of rendering modern websites in a comprehensible manner. Supporting such ancient browsers is a Bad Thing.

I’m also surprised at how hideous some of the CA websites appear: quite a few look like they haven’t been updated in at least a decade.

Lastly, there’s just way too many options presented by CAs. Domain-validated certificates are cheap and easy, though there’s no reason why phishing websites and the like can’t get perfectly-valid DV certs for their misleading or fraudulent sites: they do, after all, legitimately control their domain.

Still, DV certs provide reasonable protection from man-in-the-middle attacks, and CAs like Let’s Encrypt make DV certs available for free in an easily automated and installed way. If Let’s Encrypt’s ACME validation system won’t work for certain purposes, commercial CAs like Comodo and GeoTrust offer incredibly cheap DV certs in the form of PositiveSSL ($5/year) and RapidSSL ($9/year), respectively. Even Thawte offers relatively cheap “SSL123” DV certs for $31/year. There’s really no excuse for not using HTTPS.

Extended validation certs are useful for major companies, banks, etc. as the CA actually verifies the legitimacy of the entity behind the domain name. It should be extremely unlikely for any EV certificate to be issued illegitimately, though users might not actually check for anything more than the “green bar” (if they do that at all), so I generally think EV certs are a good idea.

That said, I’m not sure why there’s such an extreme price difference for EV certs. For example, compare Comodo ($101/year) and GeoTrust ($125/year) with Symantec ($600/year to $900/year) — the roots are equally ubiquitous and trusted, perform the same validation, and users never bother to check which CA actually issued a cert. So long as the green bar appears and the browser doesn’t yell at them, they don’t care.

Organizational and individually-validated certs are essentially worthless. They appear the same as DV certs in browser interfaces (no green bar), and essentially nobody bothers to check the O and OU fields in a certificate.

Charging more for wildcards is annoying, as it doesn’t cost the CA anything extra to issue; one of the reasons I liked StartSSL (before their WoSign-related drama) was that they only charged for things that required human action. Domain-validated certificates for non-commercial purposes are completely free of charge. OV and IV certs require a human to perform the validation, and customers pay an annual fee to be validated. Once validated, customers could issue an unlimited number of certificates — including wildcards — for any domains they controlled. EV certs were a bit different, but still quite cheap. That was a refreshing change from the business-as-usual of the CA industry, though StartSSL seem to have screwed themselves over with shady behavior after being acquired by WoSign.

Simply put, CA websites and their offerings suck. They’ve always sucked, currently suck, and likely will always suck in the future. I have no idea why such wildly-profitable organizations can’t design a website that doesn’t suck and is targeted to the relevant technical people.

Edit: It’s been brought to my attention that SSLs.com no longer offers GeoTrust, Thawte, Symantec certificates, and instead only offer Comodo certificates. I’ll keep the links here for historical purposes, but if you want to get such certificates you’ll need to find another vendor.

Mozilla preparing to remove WoSign/StartSSL from trusted root store

Last year, StartSSL, a popular Israeli certificate authority of which I myself have been a customer, was quietly purchased by WoSign, a CA in China. All well and good, such things happen fairly often in the industry.

However, they cut some corners: WoSign didn’t disclose the purchase to Mozilla, in violation of Mozilla’s policy. On its own, that’s not a super-critical issue, but that’s not all they did: based on information provided in a Mozilla report, WoSign has been caught backdating SHA1-signed certificates to avoid an industry-wide ban on that hash algorithm due to its cryptographic weakness, going so far as to provide a standardized internal framework for issuing backdated certificates. Additionally, they used the newly-acquired StartSSL to issue at least one backdated certificate.

Evidently they did this because Windows XP SP2 (a long-outdated version of XP, which is itself in end-of-life status) is quite popular in China and does not support SHA256 signatures, so there is a demand for SHA1-signed certificates. In addition, some payment processors in the US didn’t plan ahead and found some of their old payment terminals only supported SHA1 and were unprepared for the deadline and so got WoSign to backdate some new certificates to avoid any issues.

In addition, WoSign’s back-end software used for validating domains, issuing certificates, etc. has evidently had a series of bugs that have resulted in them improperly issuing certificates for GitHub and the University of Central Florida without the approval of either organization. A bug also allowed an attacker to bypass domain validation entirely and have WoSign issue certificates for unvalidated domains. While bugs are an unavoidable part of software development, such critical bugs should have been found very early in testing and never made it to production.

Their internal policies seemed geared toward “issue first, validate later and revoke if necessary”, which is absolutely the wrong way to issue certificates and which is in violation of the CA/Browser Forum Baseline Requirements.

Shockingly, WoSign’s auditor, Ernst & Young (Hong Kong), didn’t catch any of these glaring issues.

Needless to say, Mozilla isn’t happy and is discussing what to do. Right now, the most likely response is to untrust new WoSign and StartSSL-issued certificates for a period of at least one year, after which time they could reapply for trusted status by undergoing both the standard audits as well as some extra, Mozilla-specified scrutiny. Existing certificates from the CAs would continue to be recognized, but no new trusted certificates could be issued by those companies.

I find the solution to be quite elegant: CAs have occasionally played a bit fast and loose, and have relied upon their “too big to fail” status. Revoking the trust bits outright for a major CA like Symantec/VeriSign, Comodo, or even relatively smaller ones like StartSSL and WoSign, would cause a massive disruption for innocent customers of that CA and was generally only considered for the most extreme cases (see DigiNotar).

Instead, the solution proposed by Mozilla allows innocent customers to continue to use their certificates without disruption until they come time for renewal, at which point they’ll need to find some other option. The CA, however, is penalized by being unable to issue new certificates (if they do issue new certificates they’ll be untrusted, and Mozilla has threatened to blacklist the entire CA immediately if the backdates certs to avoid the restriction) and thus loses both reputation and business.

I suspect that Google, Microsoft, and Apple will follow Mozilla’s lead, so the penalty will be essentially universal.

Very cool.

Ars Technica has more details on the situation.

Personally, I’m saddened by the whole situation: other than a somewhat-clunky web interface, StartSSL had been a solid CA for years prior to their acquisition. The one black mark was their response to Heartbleed (they were charging for revoking compromised certificates) which, although in accordance with their policies, was a bit of a dick move and bad PR. I used StartSSL certs on many of my sites and had recommended them to others.

After the acquisition by WoSign (which had not been pointed out for nearly a year), StartSSL’s website switched to a poorly-translated version made in their China office (according to StartSSL). Although the speed of certificate issuance improved, the overall change was negative, with the web interface being laughably bad to use. The quality of customer service also decreased.

Still, StartSSL brought it on themselves. I no longer use StartSSL certs and don’t recommend anyone use them going forward. I may change my mind at some point in the future once they prove they’re trustworthy again, but not now.

Currently, I recommend using Let’s Encrypt, an open, automated and free CA — this site uses LE-issued certs. Installation and server configuration is automatic and easy, and renewals are handled automatically by cron job. It couldn’t be easier and I’m extremely happy.

For certain other, internal services I maintain that don’t play nice with Let’s Encrypt, I like Comodo PositiveSSL certificates sold by the reseller SSLs.com. Certs are cheap (around $5/year), issued in minutes, with a validity period up to 3 years. Unlimited reissues are included. Customer service is responsive and clueful. The one downside is their self-service interface only supports RSA certificates; if you want to use ECC certificates (Comodo PositiveSSL offers both all-RSA and all-ECC chains, which is nice) you’ll need to send the CSR to their customer service staff, who will manually submit it to the CA. They usually do this quite quickly.