One of my earliest memories of awe when learning how to program computers as a kid was wondering how the computer takes bits in the CPU and turns them into an image on a screen. When you don’t know how most of the system works, it’s easy to conclude that it is basically magic, and that the knowledge was arcane to the point of being inaccessible without tons of background in engineering and science.

As I grew up, this question never really left my mind, and really grew into the more abstract question of how a computer works at every level. Once you get into the high level of software and languages like C and Swift, you can abstract away a lot of the questions into programming language theory and ignore things like transistors and voltage. But peeling back the curtain to reveal what is going on in the circuitry lies a world that is both simple in its principles and ever complex in the myriad of tricks used to keep this wrangled chaos ticking in utter precision.

I started watching Ben Eater’s incredible YouTube series on building a computer on breadboards a couple years ago. I’ve watched the whole thing end-to-end multiple times, and have built (or tried) a couple of the modules, including the clock and the ALU. I went from knowing very little about electronics to having a fairly broad understanding of how circuits work, at least in theory.

On the other side, I’ve tried writing video game console emulators, notably a Game Boy emulator in Swift which works well enough to understand how a higher level system works with all its pieces put together. The emulator is largely unusable (hence why it doesn’t exist in an open source form) but it can run the start of a couple games, generate some graphics and sound, and process user input. Over the years I’d heard about things like Verilog and VHDL being used to define hardware in code. And in the last year or two I’ve gained an understanding of what FPGAs are (especially thanks to the MiSTer project and its focus on cycle-accurate emulation).

So in my ever expanding pursuit to learn how each piece of the computer works, I’ve picked up an FPGA development board, the DE10-Nano, and have begun using the Intel Quartus software to learn how to program it using Verilog. So far, I’ve gotten a couple small and ultra-basic projects working, namely getting an LED to blink based on an internal 50MHz clock and a clock divider, and a basic unclocked two-bit adder. As I continue going down this route, I have a number of project ideas I want to try, including:

  • emulating Ben Eater’s breadboard computer
  • a video generator that outputs something to HDMI
  • an analog to HDMI video converter and scaler, akin to the OSSC
  • an HDMI to analog video converter, so I can play PC games like Undertale on my old CRT TV
  • a Dreamcast VMU clone
  • a basic graphics card
  • a RISC-V CPU
  • some kind of module for my modular synthesizer

Will I be successful at actually building any of these? Who knows! But in the end, I’ll get a few steps closer to understanding how computers truly work at the silicon and chip level. And the journey is definitely more valuable than the destination.

Of all the companies to acquire GitHub, Microsoft is probably the best. What was a critical piece of internet infrastructure held up under a venture capital model will now at least be sustained by one of the biggest tentpole companies in the software industry. They will presumably be able to bring some organizational support and work to shore up the sites notoriously rocky reliability. And a company like Microsoft will hopefully not be able to shrug off a sexual harassment claim the way GitHub did.

I don’t see this alleviating a major problem with software engineering culture, the over-reliance on GitHub as a centralized home of code. Git is distributed by nature and most of the value added by GitHub (PRs, issues, wikis, etc.) are found on competitive platforms like GitLab and Bitbucket. But many companies rely exclusively on GitHub, and many tools like Travis CI support GitHub exclusively. Competition makes everyone better, and Microsoft will probably use its existing platforms to further lock in developers and companies and reduce competition.

I personally use a self-hosted instance of GitLab on my VPS server (which is quite easy to install nowadays), which provides me with all the features I would want and an unlimited capacity of private repositories. I use it for continuous integration and continuous deployment with its built-in Docker image registry, and those images get deployed automatically to servers. I’m hoping to do a tutorial on setting that up.

Interesting timing with WWDC kicking off tomorrow, though.

Link

Replacing Objective-C and Cocoa


Ash Furrow wrote an article arguing that Apple needed to replace Objective-C with something else. The crux of the argument is that programming languages have moved to higher levels of abstraction over time, edging further away from direct hardware access. By the time such a transition were completed (say within 10 years), using C-based languages will seem as archaic as using assembly. Ash then lays out features he would like to see in such a language.

Replacing something as fundamental to a platform as its language is no small feat. Apple did this once before with Cocoa and the compatibility bridge of Carbon when moving from OS 9 to OS X, and its migration took 12 years to be fully finished in public API. Developers fought this change for many years before Cocoa became the de-facto standard. So a migration to something newer cannot be a cavalier move done to embrace trends; it must be done with a clear purpose that fixes common issues in the thing it replaces, and it must set a foundation upon which to build at least a decade or two of software. And it must coexist with that which came before it. With the OS X transition, Apple didn’t just have a new language; they had a whole new operating system. It came with entirely different ways of handling memory, threading, files, and graphics. It delivered frameworks that were way more usable than their predecessors. It wasn’t just a new programming language; it was a revolution in how we built software.

That’s what it should take to inspire a radical change in developer tools – improvements on an order of magnitude in building software, making it easier to solve hard problems, and fixing issues in common coding standards that have arisen through heavy use. This goes beyond just a programming language; it will require new frameworks and design patterns to really bring about the benefit. Apple owns their developer technologies stack; from compilation with LLVM, to language features in Objective-C, to framework features in Cocoa, to web technologies in WebKit. When you have control of all of these pieces, the problems at the top of the stack can be addressed at the bottom, and vice-versa.

Here are some things I’d love to see in a next-generation developer platform.

Read More

In 2013, developers rely on web-based services to get much of their work done. Whether it’s looking up server API documentation, provisioning iOS devices, or registering API keys, a part of a developer’s day involves going to a website and looking something up. And that means keeping track of how to find all of these disparate websites. The problem is that there’s no real standard way or schema to get to them. Each service has a different URL for their developer site. Some examples:

There’s 10 different examples, each with completely different URL structures. If you have to interact with more than a few of these, it becomes impossible to keep track of them all in your head. Now, you could rely on bookmarks or web searches, but these aren’t as fast as just typing a URL into your browser or using a shortcut in an app like Alfred. Speaking from my own experience, for the sites I can keep in my head, I’ll type part of the URL and hope that the browser’s URL autocomplete will finish the job, simply because it’s usually more efficient. But as a human, sometimes I’m wrong, which wastes time. Wouldn’t it be great if there was a simple, consistent URL structure for all these different sites?

So I built DevCenter.me to do this. DevCenter.me is a directory of developer sites that can be accessed via consistent URLs that either you or your browser can remember. Each developer site can be accessed via one or more shortcuts (for example, ios for the iOS developer center, or app.net or adn for App.net) through either of two different URL structures: shortcut.devcenter.me or devcenter.me/shortcut. These shortcuts are designed to be memorized. Frequently used shortcuts will end up getting persisted in your web browser’s URL field (the service sends a 302 temporary redirect, not a permanent one, so your browser will probably still cache it). After a few uses, you can just type fb into your browser’s URL field, and it will automatically recognize you’re typing fb.devcenter.me, taking you to the right site in just a few keys.

DevCenter.me was designed to be a productivity tool for developers. There’s a full list of sites and their shortcuts available in JSON, as well as on the website (with shortcuts shown by pressing the option key). You can even download an Alfred extension to open a site via one of these shortcuts (shout out to Will Smidlein for writing this, as well as providing the motiviational tweets to write this service). And if a site is missing. you can fork the project on GitHub, add it to the list, and submit a pull request. When I initially published the project, there were 10 sites. As of this writing less than 12 hours later, there are 56, and undoubtedly more pull requests for me to merge in. That’s really cool to see, and I hope people keep submitting requests for more sites.

I built DevCenter.me because keeping track of all these websites was a thorn in my side; wasted memory space in my head. Accessing developer sites may not be a world-changing problem, but saving a few seconds a few times a day adds up over time. A better solution keeps you focused on what you’re actually trying to do rather than requiring you to keep estoeric URLs in your head. I hope it finds value as a tool for lots of developers.

As I’ve stated before, your app needs to sync. This has not gone unnoticed by the startup world, who are offering more and more options for developers to build sync into their apps. Today, Dropbox announced a new datastore API, a system for syncing application data (that isn’t file-based) into and out of Dropbox.

At first glance, this looks like a wonderful solution. You get a drop-in component on iOS, Android, or the web to put your app’s data into the cloud, with very little thought by you. Data gets stored offline automatically. It even handles merging and conflicts quietly in the background. Pretty great, right?

I hope that the Dropbox Datastore API can deliver on these promises. I don’t think they’re necessarily impossible problems to solve. But these are the exact same problems that Core Data + iCloud claims to solve, and between iOS 5 and iOS 6, iCloud hasn’t been able to deliver on that promise. This certainly doesn’t mean Dropbox can’t, or that the Datastore API has problems. Dropbox is certainly well versed in the concept of syncing blobs of data between multiple systems, silently and effectively, as that’s what they’ve been doing for the last five years. But this solution should be approached skeptically and carefully.

Before you ship your app built with the Dropbox Datastore, these claims should be tested thoroughly. Test data sync across 1, 2, 3, 8 different devices. Test it offline. Introduce conflicts. Save data offline. Try multiple conflicts. Create conflicts on one machine while offline, etc. There’s a lot of ways a magic sync solution can fail.

Of course, the benefits to a drop-in solution are immense. You don’t have to write sync logic. You don’t have to wake up at 4 AM because your MongoDB process randomly died, taking your server API out with it. You don’t need to handle the differences in online and offline state. You know there’s a company whose goal it is to solve this problem for you; that’s their job.

Just be careful. When it comes to any tool that claims to be a cure-all, make sure it does the job.

In the last few years, we’ve seen a pretty significant shift in how we use computers. We’ve gone from primarily using one Internet-enabled device (the PC) to using two (PC + phone) to using three (PC + phone + tablet), and who knows what else we’ll add in the next couple years. Not only are we looking up our data and documents on all these devices, we’re creating data and documents on them, and the time we’re spending to do it on the PC is getting smaller. Effortless and ubiquitous access to data is increasingly important to people.

If your app deals with user’s data, building cloud sync into your app should not be a feature you bolt on to an app – it is the feature. It’s why you will beat competitors or lose hard to them. It’s what will make your app feel effortless, thoughtless, and magical. It’s what will gain a user’s trust, and once you have that, they will sing your app’s praises and never give it up. But to earn that trust, you have to account for sync at every step of the design and engineering of your app.

Developers have a number of choices as to how to build an app around sync. You can use iCloud, you can use a hosted service like Parse, or you can build a custom sync service for your app. Each solution has trade offs. So what should you optimize for?

Read More

Dealing with dependencies in Objective-C has always been a tedious process. You typically do some git submodule stuff, import their Xcode project into yours, add a dependency, add a linker target, set some compiler flags, etc., or you include the project’s .h and .m files manually. Then you end up running into problems because the header paths are wrong, or you forgot to add some linker flags that include categories, or some other problem. If that project requires ARC or iOS 6, you have to figure that out and set it up to be consistent with your project. Then, when you need to upgrade the library, you need to make sure all these steps still work, and hope nothing new got added that might break. It’s a very error prone process. Now, being a stubborn developer that’s always done it this way, I’ve been wary of any tools to automate this process, as I usually think I can handle it myself, and I’m usually wrong. Other languages have had package managers to solve this problem, so why not Objective-C?

CocoaPods tries to solve this problem by automating the process of fetching dependencies (and recursively fetching their subdependencies), adding them to an Xcode project, managing paths for everything, adding any extra compiler or linker flags, copying in any resources (images, nibs, sounds, or whatever else), and building it into your project. The end result is a very simple process of defining your dependencies in a file (called a Podfile), running a command line process, and then just building your app and referencing those dependencies. If you need to update dependencies or add new ones, just add them to the Podfile and run the command line process again. It’s very simple, and a far cry from managing all this stuff yourself. And, as of this writing, there are over 600 projects you can include in your app.

Under the hood, CocoaPods is creating an Xcode project which builds a static library, libPods.a, consisting of all your dependencies. It adds this project to an Xcode workspace and makes your project dependent on libPods.a using an Xcode config file. It then rewrites your Xcode project to link libPods.a and copy resources, and set some paths to variables included from the config file. It even detects if your project uses ARC, and sets flags appropriately. The result is that the majority of changes to your project are minimal, but instead reference a project that is under the control of CocoaPods, and as such it can be changed while rarely affecting your project. It’s a well thought out system.

To get started, you need to install the CocoaPods gem with a gem install cocoapods at the command line. Then, in the root of your Xcode project, add a Podfile that lists your dependencies and your deployment target. For this example, we’ll target an iOS 6 app that depends on the AFNetworking and FormatterKit projects. You can search for more projects on CocoaPods.org.

platform :ios, '6.0'
pod 'AFNetworking', '~> 1.0'
pod 'FormatterKit', '1.0.1'

Note: CocoaPods uses semantic versioning to determine how to handle version numbers. The version string can either be a specific version, or can include an operator that tells CocoaPods to pick a version for you. The ~> operator says, for version X.Y.Z, “use any version matching X.Y.*”, but you can also use >, >=, <, or <= which do what you expect.

Once you have this in place, run pod install. This command will:

  • download the podspec (a manifest listing instructions on the project’s requirements and build instructions) for each dependency you list, and those for any subdependencies
  • check the requirements for each podspec to ensure that your project meets the minimum requirements (so a Mac project won’t be added to an iOS app, or a project that only works on iOS 6 will not work on iOS 5)
  • set up a new xcodeproj with a static library target for all the source files in the dependency tree
  • set up an xcworkspace if you don’t already have one
  • add the Pods xcodeproj to this new xcworkspace
  • create an xcconfigfile that includes header paths for all dependencies
  • change your xcodeproj to use the xcconfig file for header and linker paths
  • add the libPods.a library to the Link Bundle With Libraries phase of your ‘xcodeproj’
  • add a new Copy Pods Resources script phase to copy any resources to your bundle

Once this is in place, you can build and run. Unless there are any problems with the dependencies, Xcode will compile all the dependencies and link them into your app. It’s very important that you use the xcworkspace, so Xcode knows how to build the Pods project correctly. You can then #include <afnetworking/AFNetworking.h> to begin using the code. That’s it!

I’ve started using CocoaPods on a project and have been enjoying using it over managing dependencies myself. I haven’t seen any reason to believe this would be more problematic than doing it all myself, but there are plenty of benefits. Dependencies can be kept up to date much more easily, and their inclusion process is much more strictly defined (and automated). For many projects, it’s far more likely to get the setup process right than I am, and it’s faster to get set up. I recommend checking it out for your projects.

iCloud looks like it will be an incredible technology for moving app data between devices. This is inherently a good thing, and it will open avenues for many new types of apps. But, there is a fundamental problem. Right now, the only way to access it is through Objective-C APIs embedded into iOS and Mac OS X. Under the hood, they are obviously talking to the network and doing the business of syncing data, but that networking layer is not exposed or documented, and would have to be reverse-engineered in order to understand and use. So the only way for developers to move their data through this system is through a pre-compiled bundle that gets referenced within an application.

This has a few interesting practical repercussions. If you build an application targeting iCloud, you can only ever put it on two platforms – Mac and iOS. You will never be able to port it to Android, WebOS, Windows Phone, or the web (mobile or desktop). If you sync data through iCloud, And, you will never be able to have a server component that can do things with your data all the time.

Here’s some examples of what I’m talking about. In my To Do list app, Todolicious, one thing I would love to be able to do is to push badges to your iPhone and Mac showing the number of To Dos you have left. When you tap a To Do to mark it as done, suddenly all your devices would show the correct number on the icon. With the sync server I was building, this was fairly trivial; wait for the user’s list to change, and send a signal to push the count everywhere. But if I back Todolicious with iCloud, I have no way of speaking between my server and iCloud (and I’d still need a server of some kind to send the notifications, after all).

Similarly, if I were to build a web app version of Todolicious (which I was planning on), I could not get access to that data within iCloud at all. I’d have to have either to sync to both iCloud and a custom solution (unwieldy, poor UX and network traffic, and otherwise gross), or not load existing data at all (completely negating the benefit of having such a web app).

So there is a serious ecosystem lock-in problem for apps that wish to target iCloud. All of these problems go away when iCloud is made available as a server-to-server API. A big benefit in the promise of cloud computing includes service interoperability, but right now iCloud is merely a data silo. I have filed this as a bug, rdar://9598555, for a server-to-server API (through which you could build code that speaks to iCloud on your server or on other platforms). I dearly hope Apple addresses it.

Such a server-to-server API would drastically decrease the friction of setting up cloud services to complement an iCloud-backed app, and would lead to better apps and more pleased users.

Two years ago, I began working on a new Twitter client for iPhone, named Streamlines. I hinted at it about a year ago, and has been a driving force in my development of MGTwitterEngine and a ton of open source projects. I’ve come to the conclusion that I won’t have time to finish and release it, as there’s still probably another 6 months of development needed to really ship it, and hostility from Twitter and from users of other Twitter clients make effort into building one unsustainable. However, I think there are UI concepts in there which are totally unique and have never been seen before, so I’d like to share them with you before this project is lost to the annals of dead projects.

Read More

Twitter uses OAuth as its supported form of authentication. This is fine for some apps like clients where users need to authenticate themselves, but it doesn’t work well for bots or scripts run by one person. If the bot needs an access token, it can be a real pain to obtain without writing the intermediate code.

This application, OAuthery, serves a simple purpose. You supply it the consumer key and secret for your OAuth application. It creates a request token and the URL for authorizing it. You complete the authorization in a web browser and get a PIN number. You then enter that PIN number back into the application, and it spits out your access key and secret. At this point, the user authentication is complete, and you can add those credentials to your script, and access API resources.

This is largely intended as a developer tool for people developing applications with Twitter’s OAuth impelmentation. It also provides code to show exactly how to perform the authentication process with OAuth and Twitter, so that developers who wish to implement such functionality in their own apps can use this as reference.

Link

OpenEmu is a Cocoa framework and application for running multiple emulators as plugins. Many popular open source emulators; such as SNES9x, Nestopia, and VisualBoyAdvance; are fully compatible with this system. As all the emulators are going through the same architecture, it can be tuned to use the latest Quartz and Core Graphics technologies to deliver screaming performance.

Most of my participation has been in fixing bugs and doing some application-side coding. My big task so far is an in-development ROM organizer powered by Core Data, with smart playlists, Quick Look for save states, and ratings.

Link

VillainousStyle is a drawing library for defining a visual style from a chain of individual drawing instructions. Each instruction modifies the drawing context to perform common operations; such operations include shadows, fills, borders, and shapes. It allows for multiple style sheets which can be used to theme an application in multiple visual contexts. VillainousStyle sits on top of CoreGraphics, and does not use WebKit for rendering at all. It is a fork of the VSStyle and VSShape classes, originally from the Three20 project.

Stylesheets

VSStyleSheet is an abstract superclass for a set of styles. Subclass it and add methods for each style you wish to add. You will likely want to create a protocol for your styles to implement, to ensure that your stylesheet implements all the necessary styles.

There is a global stylesheet, which can be thought of as the “active” stylesheet. Call +[VSStyleSheet setGlobalStyleSheet:] to change the active theme, which will fire a VSStyleSheetChangedNotification. When that gets fired, you’ll want to tell your views to update their styles and redraw.

Styles

Styles affect drawing and positioning. Most will affect the next VSStyle objects in the chain.

  • Fills
    • VSSolidFillStyle – Fills the current shape with a solid color
    • VSLinearGradientFillStyle – Fills the current shape with a gradient between two colors
    • VSReflectiveFillStyle – Fills the current shape with a glossy-style gradient between two colors
  • Borders
    • VSSolidBorderStyle – Draws a border around the current shape with a solid color
    • VSBevelBorderStyle – Draws a beveled edge border for a 3D effect around the current shape
    • VSFourBorderStyle – Draws a border around the current shape with four colors, one for each edge
  • Shadows
    • VSShadowStyle – Draws a shadow behind content with a given color, blur, and offset
    • VSInnerShadowStyle – Draws a shadow inside the content with a given color, blur, and offset
  • Positioning
    • VSBoxStyle – Adds a margin or padding to the content area
    • VSInsetStyle – Adds edge insets to the content area
  • Content
    • VSTextStyle – Draws text inside the current shape
    • VSImageStyle – Draws an image inside the current shape
    • VSMaskStyle – Clips the drawing area to an image mask
    • VSShapeStyle – Clips the drawing area with a VSShape object

Shapes

Shapes affect the fills and borders, but do not clip the content styles.

  • VSRectangleShape
  • VSRoundedRectangleShape
  • VSRoundedLeftArrowShape – a rounded rectangle with a left-facing arrow
  • VSRoundedRightArrowShape – a rounded rectangle with a right-facing arrow

Future Ideas

  • iPhone static library
  • Cappuccino library
  • File-based stylesheets that can be read/written from VSStyleSheet objects
  • GUI builder for styles
  • More styles!
Link

The templates that ship with Xcode are not the greatest. Some of them are inconsistent and don’t enforce good coding standards (e.g. missing a dealloc method). Other templates which would be useful just flat out don’t exist (e.g. an NSOperation subclass, or a protocol header file). This project aims to supplement or replace the built-in templates for Xcode to speed up coding and improve the quality of code.

Coding Standard

All files will be processed by Xcode. The generated source files must produce consistent, readable, commented code. The code must have these characteristics:

  • Each file must have a comment block at the top describing the file.
  • Each class must implement its superclass’ designated initializer and dealloc.
  • Stub methods must be organized by their purpose, class or protocol. — Each group must be organized by their class hierarchy, with protocol stubs following. — Each group must be prefaced by a pragma mark naming the class or protocol the methods were implementing. — Clusters of methods (such as relating to KVO) should be organized along the lines above, with a pragma mark.
  • All method implementations should contain a method call to their super implementation if needed.
  • All method implementations should contain a commented out stub line that will signify where to insert their code.
  • All comments must be in the form of two slashes //, and none using the /* */ form. This will allow developers to comment out large blocks of code as needed.

Wish List

  • Different people want different things in their template. For instance, someone may want to include an implementation of observeValue:… for every class. Would be nice to have a template generator application (yeah yeah, very meta) which would make templates customized to the developer.
Link

URL Shrink is a new OS X tool with a very simple purpose – converting URLs to shorter permalinks on various web services. As the internet has matured, and services like IM, IRC, and Twitter have forced us to write shorter messages, it was clear that a system was needed that was as ubiquitous as Quicksilver.

For now, the main service of URL Shrink is just converting a URL that is on the clipboard, although this will be expanding over time (including things like a system text service, a command line client, a Quicksilver plugin, etc.). To do this, there is a keyboard shortcut (option-shift-space) which will convert the URL in the background to a tiny URL using one of the services provided. If you’ve selected one as your favorite, it’ll choose that one. For example, I’m personally partial to is.gd, so all the URLs that are processed by URL Shrink on my machine go through is.gd.

At a low level, URL Shrink is a system where multiple shrinking engines can be added. It was designed to be extremely easy for developers to write just a little bit of code to integrate with other web services, including private URL services. For information on that, see the project page below.

URL Shrink is licensed under the BSD license. I encourage its adoption within other applications; I’ll be adding .framework and .a targets for building this into Mac and iPhone apps respectively. Indeed, the project was born out of the URL shrinking capacity of another app I’m working on.

Link

This one is for all of those developers out there who scoff at JavaScript. This is a working neural network algorithm in JavaScript used for ripping apart CAPTCHA images (in this case, from Megaupload) and deciphering them. This is really sophisticated stuff, and even though Megaupload has some pretty easy CAPTCHAs, this should be pretty easily adaptable to other CAPTCHAs.

Link