{"id":118,"date":"2009-08-27T08:10:34","date_gmt":"2009-08-27T07:10:34","guid":{"rendered":"http:\/\/lendl.priv.at\/blog\/2009\/08\/27\/cryptopensslx509-and-utf-8-characters\/"},"modified":"2026-01-26T12:16:12","modified_gmt":"2026-01-26T11:16:12","slug":"cryptopensslx509-and-utf-8-characters","status":"publish","type":"post","link":"https:\/\/lendl.priv.at\/blog\/2009\/08\/27\/cryptopensslx509-and-utf-8-characters\/","title":{"rendered":"Crypt::OpenSSL:X509 and UTF-8 strings"},"content":{"rendered":"

Bumped to top due to updates.<\/em><\/p>\n

For my current project I look at a lot of X.509 certificates<\/a> using Dan Sully’s Crypt::OpenSSL:X509<\/a> Perl module. I’m not using the version from CPAN, but his current codebase straight from his git repository<\/a>.<\/p>\n

While trying to store information about certs in a PostgreSQL DB which is set to UTF-8 strings, I encountered errors. Some debugging later I found that some of the certs had Umlauts in the subject field. The XS code from Crypt::OpenSSL:X509 wasn’t UTF-8 aware, causing automatic down-conversion to ISO-8859-1, which produced illegal byte sequence when parsed as UTF-8.<\/p>\n

After some cursing and debugging I came up with this patch<\/a>:<\/p>\n


\n--- ..\/dsully-perl-crypt-openssl-x509\/X509.xs 2009-03-06 22:22:44.000000000 +0100
\n+++ X509.xs 2009-08-17 14:46:00.000000000 +0200
\n@@ -73,6 +73,15 @@
\n return sv;
\n }<\/p>\n

+static SV* sv_bio_utf8_on(BIO *bio) {
\n+
\n+ SV* sv;
\n+ sv = (SV *)BIO_get_callback_arg(bio);
\n+ SvUTF8_on(sv);
\n+ return sv;
\n+}
\n+
\n+
\n \/*
\n static void sv_bio_error(BIO *bio) {<\/p>\n

@@ -293,8 +302,10 @@
\n name = X509_get_issuer_name(x509);
\n }<\/p>\n

+ \/* this need not be pure ascii, try to get a native perl character string with utf8 *\/
\n+ sv_bio_utf8_on(bio);
\n \/* this is prefered over X509_NAME_oneline() *\/
\n- X509_NAME_print_ex(bio, name, 0, XN_FLAG_SEP_CPLUS_SPC);
\n+ X509_NAME_print_ex(bio, name, 0, (XN_FLAG_SEP_CPLUS_SPC | ASN1_STRFLGS_UTF8_CONVERT) & ~ASN1_STRFLGS_ESC_MSB);<\/p>\n

} else if (ix == 3) {<\/p>\n

@@ -799,7 +810,8 @@
\n n = OBJ_nid2sn(nid);
\n }
\n BIO_printf(bio, \"%s=\", n);
\n- ASN1_STRING_print(bio, X509_NAME_ENTRY_get_data(name_entry));
\n+ sv_bio_utf8_on(bio);
\n+ ASN1_STRING_print_ex(bio, X509_NAME_ENTRY_get_data(name_entry),ASN1_STRFLGS_UTF8_CONVERT & ~ASN1_STRFLGS_ESC_MSB);
\n RETVAL = sv_bio_final(bio);<\/p>\n

OUTPUT:
\n<\/code><\/p>\n

Basically, this just tells the openssl library to output UTF-8 and the perl core that the new strings are encoded in UTF-8.<\/p>\n

This might be overkill in the cases where it’s not actually needed, but it should do no harm.<\/p>\n

Update:<\/strong> My patch is now in the git repository<\/a>.<\/p>\n

Update2:<\/strong> Life is not that easy. Looking at more X.509 certs in the wild shows that openssl does not check whether it returns a valid UTF8 string. So stay tuned for additional patches in Dan’s git repository.<\/p>\n

Update3:<\/strong> My patches are now integrated in the git version<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Bumped to top due to updates. For my current project I look at a lot of X.509 certificates using Dan Sully’s Crypt::OpenSSL:X509 Perl module. I’m not using the version from CPAN, but his current codebase straight from his git repository. While trying to store information about certs in a PostgreSQL DB which is set to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,11],"tags":[],"class_list":["post-118","post","type-post","status-publish","format-standard","hentry","category-cert","category-system-administration"],"_links":{"self":[{"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/posts\/118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/comments?post=118"}],"version-history":[{"count":1,"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/posts\/118\/revisions"}],"predecessor-version":[{"id":981,"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/posts\/118\/revisions\/981"}],"wp:attachment":[{"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/media?parent=118"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/categories?post=118"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lendl.priv.at\/blog\/wp-json\/wp\/v2\/tags?post=118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}