Karl von Randow

Charles Proxy now available on iOS

Wed, 28 Mar 2018 12:50:00 +1300

We are excited to announce that Charles Proxy is now available on iOS!

With the iOS version of Charles you can capture and inspect network requests and responses on your iOS device. You can view metadata, headers and bodies in the app, so you can finally debug your app’s networking issues without a computer.

Charles for iOS currently supports the following features:

Capture HTTP and HTTPS network traffic on your device
SSL Proxying so you can view your SSL / TLS requests in plain text
View requests and responses in the app, or share each request or a whole session to Charles on your desktop

Running Charles on your iOS device means you no longer need to fiddle with WiFi network proxy settings. It also means that you can capture and measure network traffic that goes over the Mobile / Cellular data network.

Measuring networking performance over Mobile data is especially important for your mobile apps (as that is how a lot of users experience your app), and it can reveal large or slow requests, as well as opportunities to increase perceived performance by parallelising network calls.

We are thrilled to be able to finally bring Charles to the App Store. If you’d like to know a bit more about the history of Charles, and particularly Charles on iOS, I gave a presentation to try! Swift Tokyo in March 2018 that you can watch on YouTube.

We plan to bring more of the features that you know and love from Charles on the desktop to the iOS version over the coming months, so watch this space for more announcements.

We hope you enjoy using Charles on iOS!

How do I download an iOS App (IPA) file to my Mac after iTunes 12.7 update?

Thu, 07 Dec 2017 18:52:25 +1300

How do I download an iOS App (IPA) file to my Mac after iTunes 12.7 update?

How To Shrink Your Virtualbox VM And Free Up Space For Your Hard Disk - Make Tech Easier

Tue, 25 Apr 2017 08:29:54 +1200

How To Shrink Your Virtualbox VM And Free Up Space For Your Hard Disk - Make Tech Easier

Roadwarrior configuration for macOS 10.12, iOS 10 and Windows 10 using strongSwan and user certificates

Thu, 30 Mar 2017 16:15:40 +1300

Roadwarrior configuration for macOS 10.12, iOS 10 and Windows 10 using strongSwan and user certificates

GeoIP on PostgreSQL 9.5

Mon, 23 Jan 2017 17:53:56 +1300

This guide is for adding GeoIP functionality to PostgreSQL 9.5 (and earlier) using MaxMind’s GeoLite2 CSV files.

You can read about the MaxMind GeoIP2 CSV files, and download free versions. You’ll need the CSV files to continue.

I create two tables; geoip_blocks and geoip_locations to contain the data. I load the English country names, but you could choose the language you want when you populate the geoip_locations table. I have copied the relevant GeoLite2 CSV files into the /tmp directory in the example below, but you could read them from anywhere.

You can use the index support for cidr addresses built in to PostgreSQL 9.4 and later, but you get much better performance using ip4r instead.

I have examples of both approaches here: https://gist.github.com/karlvr/8ff1900bfc9cba36468640c26ae3b2bb

AWS CloudFormation UserData for Ubuntu 16.04 LTS

Sat, 14 Jan 2017 08:17:24 +1300

I use CloudFormation to create EC2 instances on AWS. I use this UserData script to install the CloudFormation scripts on the EC2 instance and then to run the init and signal steps.

"UserData": {
"Fn::Base64": {
"Fn::Join": [
"“,
[
”#!/bin/bash -x\n",
"apt-get update\n",
"apt-get install -y python-pip\n",
"pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n“,
"cfn-init -v –resource <RESOURCE NAME CONTAINING INIT CFG>”,
" –stack “,
{
"Ref”: “AWS::StackName”
},
" –region “,
{
"Ref”: “AWS::Region”
},
"\n",
"cfn-signal -e $? –resource <RESOURCE NAME CONTAINING CREATION POLICY>“,
” –stack “,
{
"Ref”: “AWS::StackName”
},
" –region “,
{
"Ref”: “AWS::Region”
},
"\n"
]
]
}
}
}

You then need to add the CloudFormation init configuration to the resource named above. I put it in the instance resource, but I don’t think it has to be.

For example:

“Metadata”: {
"AWS::CloudFormation::Init": {
"config": {
“packages”: {
“apt”: {
“awscli”: []
}
}
}
}
}

You can also add a CreationPolicy onto a resource; usually the instance or the auto-scaling group. Creation policies have replaced WaitConditions.

"CreationPolicy": {
"ResourceSignal": {
"Timeout": “PT5M”
}
}

AWS VPC VPN Strongswan configuration

Sat, 14 Jan 2017 08:07:08 +1300

Create the VPN Connection in the VPC Management console on AWS, using static routing, then download the Generic configuration. The downloaded text file contains some values that you’ll need. There are two VPN configurations in it. I just hook up one on the server. Perhaps if you have two VPN servers you could set up one VPN on each.

These are the values of interest in the downloaded text file:

Pre-Shared Key
Outside IP Addresses

Customer Gateway
Virtual Private Gateway

Inside IP Addresses

Customer Gateway
Virtual Private Gateway

My server has an internal IP address, and sits behind a router, which has a public IP address. AWS VPC supports NAT-T so this is no problem. You just set “left” (below) to your internal IP and “leftid” (also below) to your public IP.

Here is the example /etc/ipsec.conf:

conn vpc
mobike=no
type=tunnel
compress=no
keyexchange=ikev1
ike=aes128-sha1-modp1024!
ikelifetime=28800s
esp=aes128-sha1-modp1024!
lifetime=3600s
rekeymargin=3m
keyingtries=3
installpolicy=yes
dpdaction=restart
authby=psk
left=<ip address of your server>
leftid=<public ip address of your server>
conn vpc1
also=vpc
auto=add
right=<Outside IP Addresses: Virtual Private Gateway>
leftsubnet=<your subnet>
rightsubnet=<VPC subnet>
conn vpc1a
also=vpc
auto=add
right=<Outside IP Addresses: Virtual Private Gateway>
leftsubnet=<Inside IP Addresses: Customer Gateway>
rightsubnet=<Inside IP Addresses: Virtual Private Gateway>

Here is the example /etc/ipsec.secrets:

<Outside IP Addresses: Virtual Private Gateway> : PSK “<Pre-Shared Key>”

Then in your AWS VPC configuration edit the route table and add a static route for your internal subnet to the Virtual Gateway device.

Check your security groups on your instances to make sure they allow connectivity from your internal subnet IPs. It can be useful to allow ICMP so you can test using ping.

Restart Strongswan:

service strongswan restart

Then try to bring up the VPN interface:

ipsec up vpc1

If all is going well you should see a successful connection result in a second or two. If not, something is wrong :-(

Try to connect to one of the servers in your VPC. If you can’t, check the security groups on them, or perhaps any firewall rules on your own machine.

Then bring up the vpc1a connection. This should result in the VPN showing as UP on the AWS VPC VPC Connection configuration page.

Once you’re happy, change “auto=add” to “auto=start” in /etc/ipsec.conf and restart Strongswan and the VPN would come up automatically.

Cocoapods and Swift 3

Sat, 03 Dec 2016 10:10:51 +1300

Cocoapods supports Swift 3 as of 1.1.0. Cocoapods sets the SWIFT_VERSION of the pod targets to match the SWIFT_VERSION set on your project. This is a pretty clever solution to Swift evolution, and you can read more about it in the Cocoapods 1.1.0 announcement blog.

Note the bold text on project in the previous paragraph. You need to set the SWIFT_VERSION Build Setting on your project, not just on your project’s targets.

Setup exim4 and DKIM

Sat, 26 Nov 2016 12:47:54 +1300

I have setup exim4 to sign outgoing email using DKIM, with the goal of improving deliverability. I have setup exim4 to handle multiple outgoing domains with different keys.

The key to use is chosen based on the domain name of the From header, not the envelope mail from. This is because I believe DKIM verification uses the From header (https://dmarc.org/wiki/FAQ).

I based my approach on instructions from https://debian-administration.org/article/718/DKIM-signing_outgoing_mail_with_exim4, but they appeared to be either slightly incorrect or out of date.

Configure exim4

First you need to make up a DKIM selector. This is the part of the domain name that goes on the front of the ._domainkey.example.com DNS entry you’ll later make. I configure one DKIM selector for all of the domains, so I choose one that identifies the box or the entity that owns the box. If you set up DKIM for the same domain on another box with different keys, the selector is what identifies which keys to use.

Say you choose the selector “mydkim”.

If you have split configuration, edit /etc/exim4/conf.d/main/00_exim4-local_options and append the following to the end of that file. If you are not split, edit /etc/exim4/exim4.conf.template and insert the following after the MAIN CONFIGURATION title comment near the top of the file.

DKIM_CANON=relaxed
DKIM_SELECTOR=mydkim
DKIM_DOMAIN = ${sg{${lc:${domain:$h_from:}}}{^www.}{}}
DKIM_FILE = /etc/exim4/dkim/$dkim_domain.pem
DKIM_PRIVATE_KEY = ${if exists{DKIM_FILE}{DKIM_FILE}{0}}

Run update-exim4.conf to update your configuration. Restart exim4 and check that you can still send email. I suggest sending email to another email account, not on your server. Check that /var/log/exim4/paniclog remains empty after successfully sending email.

Congratulations you’ve successfully setup exim4.

Exim4 will look in /etc/exim4/dkim/DOMAIN.pem for a private key to use to send email with a From address domain of DOMAIN.

Generate DKIM keys

Now you need to generate keys for exim4 to use. In the example below, replace “example.com” with your domain name.

cd /etc/exim4/dkim
openssl genrsa -out example.com.pem 1024 -outform PEM
openssl rsa -in example.com.pem -out example.com.pub -pubout -outform PEM

Don’t send any email yet. Exim4 will start using this private key immediately, and it will sign your email. But you haven’t setup your DNS yet.

Add DKIM public key to your DNS

You need to add your DKIM public key to your domain’s DNS. This is how a remote server can lookup the public key to check the signature your server has added to your email.

Open the example.com.pub file generated above.

Now if you send email to an email account not on your server, you should see a Dkim-Signature header in it. It will start with —–BEGIN PUBLIC KEY—– and end with —–END PUBLIC KEY—–. The line in between are your public key!

Create a new DNS record on your domain using the DKIM selector you chose above. In my example this is mydkim, so the record we add is for mydkim._domainkey.example.com.

The DNS record type is TXT. You should already have an SPF record of type TXT. If you don’t, google SPF records.

The value of the TXT record starts with:

v=DKIM1; k=rsa; p=

Directly after the last =, paste each line of your public key with no new lines. So it all ends up on one line.

e.g. v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDAPUE4hEV7wSh7v9s/NrhbJu7k1/Jqr3Mt7pspT6o7c/Q+GFP1Ko7VA0I12RYB+PeFUE+3i3yu1fsmzGn92GdhKd2nObZbs06Rynm48yVPmzXV2pEptebfOTfAdsJh3rryB0HQnPx+H1gKww1/nUagYlUjktBL7sRDGdjNqTIxYwIDAQAB

Save that DNS record.

If you want to verify that it worked, try this command-line tool:

dig txt mydkim._domainkey.example.com

Of course you probably need to wait for your DNS to update. If you know the primary DNS server for your domain, try querying that directly if you’re impatient.

Testing

Now exim4 is signing your email, and you’ve setup your domain, so it’s time to send a test email.

Send an email to check-auth@verifier.port25.com and in a few seconds you’ll receive an email back summarising the findings about your email. With luck you’ll have passed the DKIM check. If the DKIM check failed, scroll down to read why. If it is because the _domainkey subdomain we added doesn’t exist, perhaps you need to wait a little longer for the DNS to propagate. Check though that it reports the same subdomain that you added! This will confirm that you have set your DKIM selector correctly.

Now send an email to yourself on another email server, e.g. a gmail address. Check that the received email contains a Dkim-Signature header. That’s the magic bit.

Adding additional domains

When you want to add DKIM signing for additional domains, repeat the Generate DKIM keys, Add DKIM public keys to your DNS and Testing steps. You don’t need to configure exim4 anymore as it’s setup for as many domains as you like.

How to prevent updating of a specific package?

Wed, 23 Nov 2016 09:55:06 +1300

How to prevent updating of a specific package?:

Hold a package to prevent it from being upgraded unless you specifically upgrade it. I use this to hold postgresql packages, as installing upgrades results in a restart of the database server and a micro-outage that I want to better manage.

apt-mark hold postgresql-9.5

Then the postgresql and postgresql-contrib (and sometimes client) packages appear as “held back” when you run apt-get upgrade.

You can force them to upgrade when you’re ready by explicitly installing: apt-get install postgresql-9.5

Setup Ruby on macOS without sudo

Sat, 13 Aug 2016 09:18:26 +1200

Setup Ruby on macOS without sudo:

Setup ruby on macOS without sudo and with the latest version!

It’s clear that using sudo to install gems is a bad idea. And you’re stuck with an older version of ruby and gem. But is it a lot of work to change your ways? I thought so. But it isn’t.

First I nuke my gems from orbit (it’s the only way to be sure). Some of the gems are default gems so we can’t uninstall them and you might see error messages for them.

for i in $(gem list | awk '{print $1}') ; do gem uninstall -a -I "$i" ; done

After this, gem list returns nothing, or just the default gems.

Then you can install rbenv, and a new version of ruby in its own little environment just for you, so no sudo:

brew install rbenv ruby-build

# Add rbenv to bash so that it loads every time you open a terminal
echo 'if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi' >> ~/.bash_profile
source ~/.bash_profile
# Install Ruby
rbenv install 2.3.1
rbenv global 2.3.1
ruby -v

Then you’re gem installing with no sudo 4eva. Of course you need to reinstall all the gems you’ve lost. I found it rather refreshing as I didn’t really know whether I still needed any of the ones I had installed!

Charles 4

Tue, 02 Aug 2016 08:15:00 +1200

At long last, Charles 4 has been released! Charles 1 was released in 2002, nearly 14 years ago. Since then we’ve been through Charles 2 and Charles 3, both big upgrades. But for the past 11 months we’ve been hard at work on the biggest upgrade we’ve ever made to Charles!

Charles 4 includes an upgrade to the latest and greatest HTTP standard, the latest and greatest IP standard, and the biggest UI update ever for Charles. It is also the first paid upgrade in Charles’s long history.

HTTP 2 is all around you. It’s sneaky and you might not have noticed. Both the client and server need to support HTTP 2, otherwise they will downgrade automatically to HTTP 1.1. So with Charles 3 you would just have seen HTTP 1.1 everywhere. No more! With Charles 4 you can now see HTTP 2 working, and you can use all of your familiar tools; Repeat, Breakpoints, and so on. You’ll spot HTTP 2 hosts in Charles as they use a different icon—with a lightning bolt!

IPv6 is also all around you! If you’re lucky enough to have an IPv6 network and ISP you can turn on “Prefer IPv6 addresses when connecting to dual-stack hosts” on the Options tab in the Proxy Settings. Then you’ll see IPv6 addresses being used in Charles. Feel the modern IP networking (first published in 1998) flow through you!

With Charles 4 we’ve also taken the opportunity to spruce up the look of Charles, starting with the familiar, enigmatic and much-loved app icon.

I’ll let that sit by itself for a moment.

Fear not! It’s still the Charles jug, we’re not crazy! It’s just had a big polish, and we think it’s more beautiful than ever! We hope you do too.

We’ve also designed a completely new custom set of icons, giving Charles a more unified and refined appearance. Thank you to the wonderful Wolfgang Bartelme for all of his painstaking work on these icons and on our beloved Charles Jug 4.0.

Finally we’ve improved the native platform stylings on Mac OS X (soon to be macOS) and Windows; we’ve reduced border chrome and tweaked most of the dialog layouts in the app so that Charles looks more modern than ever.

On Mac OS X we’re using the brand new VAqua library by the excellent Alan Snyder. VAqua provides better graphical accuracy and higher performance, especially on retina screens. The keenest eyes amongst you may also notice that Charles 4 now uses sub-pixel antialiasing! Zoom in to check it out. And on the latest versions of Mac OS X, Charles 4 uses the new San Francisco typeface. Finally, for my Charlesitans with dark menu bars—there’s a little something for you in Charles 4 too.

On Windows we’re using the latest version of JGoodies, with support for Windows 8 and Windows 10. No more will your menus be odd colours! We’ve also fixed the egregious aliased text that previously appeared whenever you looked at XML responses.

There are lots of other little changes and improvements in Charles 4. We’ll be documenting these and updating the website over the coming weeks. Reach out to us on Twitter @charlesproxy if you have any questions or comments!

Don’t forget to download Charles 4 and check it out!

How to install a specific version of a ruby gem?

Thu, 14 Jul 2016 15:10:07 +1200

How to install a specific version of a ruby gem?:

Say you need to run the pre 1.0 Cocoapods?

sudo gem install cocoapods -v 0.39.0
pod _0.39.0_ install

Disable documentation generation for gem update

Tue, 12 Jul 2016 15:49:41 +1200

Disable documentation generation for gem update:

gem update is the slowest thing my computer does. This makes it not so slow. Put it in your ~/.gemrc and never look back:

gem: –no-document

letterboxd: App. This week we launched our Letterboxd iPhone...

Sun, 20 Mar 2016 10:08:12 +1300

letterboxd:

App.

This week we launched our Letterboxd iPhone app as a free download through Apple’s App Store. We’re ecstatic to put this app in your pocket (18,000 of you have grabbed it already!), and tremendously excited about how it will evolve the way we use Letterboxd. This was a monumental achievement for the team at HQ (particular hat tips to Grant, Ryan and Karl); the scope of the API + app development turned out to be comparable to the amount of work we did in 2011 while preparing our initial public beta.

Download from the App Store

The launch was covered by TechCrunch, IndieWire, The Next Web and many others, and the response from those who’ve installed it has been overwhelmingly positive. Today I’d like to share a little insight into the process and tell you about what we’ve built.

Process

After committing to build an iPhone app for Letterboxd, we first needed to add an API layer to our server infrastructure, to handle the transfer of data between external clients and the site. We’d piloted a small portion of the required work for Film Fest Buzz, and over the course of the past year we’ve built out the remainder of the API, which today comprises 40 individual endpoints.

The app’s interface represents an iOS-friendly take on the site’s aesthetic, and some parts of it may influence the future direction of the site. The majority of the app’s screens were designed in late 2014 — our prototype extended to 63 individual layouts, a large proportion of which have made it into the app we shipped this week.

API and iOS development was done concurrently for much of last year, and in November we invited Patron members to install and test an alpha version of the app. As always, we’re indebted to everyone who helped us to test the various builds over the past four months. (If you’ve been testing the app, please install the App Store version and we’ll get in touch when there are future betas to look at.)

What can it do?

Version 1.0 of the iPhone app supports:

Sign in and account creation (with 1Password support)

Browsing of popular, highly rated and most anticipated films

Film info including poster, trailer, cast, crew, studio and genre data, plus popular lists and reviews

Film actions: rate, like, mark as watched, add to watchlist, add/remove from a list

Logging of films with date, rating, review and tags (plus flags for spoilers and rewatches, and Facebook sharing if your account is connected via the site)

Viewing (and filtering) of your activity feed

Viewing and commenting on lists and reviews

Viewing of member profiles

Following and blocking of members

Searching for films, reviews, lists and people

Profile editing (including favorite films)

Reporting of content and members

What can’t it do (yet)?

Some items from our wishlist didn’t make it into the first version but will be added in future updates. These include:

More browsing options, including decade/year and genre

Sorting and visibility filters

List creation/deletion, editing and re-ordering

Availability on streaming services

Detail views for more screens (such as watched films and liked content)

Completion percentages for lists and cast/crew

And there’s some OS-level integrations we plan to add as well:

Notifications for follows, likes, comments, etc.

A URL scheme so the app is able to open relevant web links natively

3D Touch support

Reactions

We asked on Twitter and Facebook for your reactions to the app news, and boy did you come through with flying colors. The three recipients of a year’s Pro subscription are Jennifer MacDougall for this delightful depiction of app love, Pedro Strazza for Jonah Hill’s delayed fist pump, and poor Niall Murphy who expressed his disappointment on behalf of the entire Android community in a single word. (“[We]’ve never seen such devotion in a droid before.”)

SSL

The API requires all communication with our servers to be made over an SSL connection. Prior to the launch of the first alpha version last year, we also enabled SSL as an option for site traffic (try it). We intend this to become the only way to access the website as well.

What’s next

As alluded to above, we weren’t able to make this a dual release with an accompanying Android version. We’ll be putting some focus on the public API and Android next (we’ll let you know as soon as we have something to announce), along with more site and app improvements. And if you have any app feedback, we’d love to hear it through the usual channels.

varnishlog queries

Wed, 16 Mar 2016 15:55:52 +1300

I always have to look up how to do queries on varnishlog. This is a good link for brief docs on the query language: https://www.varnish-cache.org/docs/trunk/reference/vsl-query.html

varnishlog -q ‘ReqURL ~ “^/about/” and ReqHeader:User-Agent ~ “Safari”’

How to Downgrade a Package via apt-get?

Wed, 16 Mar 2016 15:51:22 +1300

How to Downgrade a Package via apt-get?

Keychain Access: This certificate has an invalid issuer

Wed, 24 Feb 2016 23:01:10 +1300

Keychain Access: This certificate has an invalid issuer

Debugging Makefiles

Tue, 23 Feb 2016 08:29:18 +1300

Debugging Makefiles:

Particularly $(warning Say some stuff) or $(warning Value of $(this_variable)).

Solve screen error "Cannot open your terminal '/dev/pts/0' - please check" - makandropedia

Mon, 08 Feb 2016 16:11:17 +1300

Solve screen error "Cannot open your terminal '/dev/pts/0' - please check" - makandropedia