Edenwaith Blog

Permanent Eraser 2.7.3 is now available from this website and the Mac App Store (MAS). The MAS version's release was delayed for a few days as Apple's review process was not available for a week during the holidays. The following are the changes in this version:

• Fixed default Preferences for new users.
• Minor UI fixes.
• Updated German localization.
• Updated Help files.

Since version 2.7.2 was released last November, Permanent Eraser was also updated so it could be approved for the Mac App Store. The previous version in MAS was 2.5.3, which encountered a severe issue when working with macOS Sierra and later. This issue had been fixed years earlier in version 2.6.0, but since the MAS version of Permanent Eraser had not been updated since then, that version was essentially broken and useless on newer versions of macOS.

While working on the MAS update for Permanent Eraser, I encountered a couple of additional areas which needed to be updated. Since the major planned feature for version 2.8.0 will not be accepted for the Mac App Store, 2.7.3 was created to get in these minor updates for both the website and MAS versions. Permanent Eraser 2.8.0 is planned for release later in 2018.

Many of the plans for Edenwaith in 2018 are a continuation of what was started in 2017. EdenList 2.0, a complete rewrite of this app, is still in the works. Permanent Eraser 2.7.3 was released a few weeks ago (the Mac App Store version is still waiting for approval from Apple), and Permanent Eraser 2.8.0 is also in the works. Eventually the entire Edenwaith website will be redesigned with a new appearance, better support for mobile devices, and a reorganization of the content. Some old material will finally be archived or removed, and other pages will be simplified (mostly for historical purposes for retired projects).

Augmented Reality (AR) has been around for a number of years, but it has only been in the past year that AR has finally been making some inroads into the mainstream, starting with the mobile game Pokémon GO.

Now Apple is opening up the capabilities of AR to developers and millions of AR-ready iOS devices with the introduction of the new ARKit framework. Developers have quickly embraced the new capabilities provided by ARKit by developing useful utilities to games enhanced by the AR experience.

There are numerous articles currently available about how to initially set up an ARKit project, so this post will focus more on specific topics when developing with ARKit and SceneKit.

This article makes use of a sample AR demo project which detects a plane, loads in a 3D model of a dragon, places the model on the plane, and then animates the dragon when it has been tapped.

Plane Detection

One of the key aspects to AR is for the device to be able to inspect its environment so it can learn how to interact with its surroundings, especially when trying to place virtual objects on a flat surface. Since ARKit does not come with a Hervé Villechaize module, your AR app will need to implement the ARSCNViewDelegate to help find "da plane".

Plane detection is initially disabled, so it needs to be set, otherwise the device will not look for available surfaces. To enable plane detection, ensure that the ARWorldTrackingConfiguration object's planeDetection property has been set to .horizontal.

// Create a session configuration
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = .horizontal
configuration.isLightEstimationEnabled = true

// Run the view's session
sceneView.session.run(configuration)

ARKit currently only supports the detection of horizontal planes, but there is the potential of vertical plane detection in a future version of iOS.

Plane detection is far from a precise science at this point, and it usually takes at least several seconds to detect a suitable plane. You might need to move your iOS device around so it gains knowledge of its environment so it can better estimate the distance to surrounding objects.

To aid in detecting a plane, set the sceneView.debugOptions = [ ARSCNDebugOptions.showFeaturePoints ] to provide the yellow dots, which indicates that the camera is trying to detect reference points in the environment. Objects which are shiny or lack any proper definition make it difficult for the device to obtain a decent reference point and to be able to distinguish unique points in the environment. Areas with poor lighting conditions can also compound these problems. If you are not seeing many yellow feature points, slowly move around the area and point the device's camera at different objects to help determine which surfaces can be identified.

Once a plane is detected, the ARSCNViewDelegate method renderer(_:didAdd:for:) is called. In this example, we check if the argument anchor is an ARPlaneAnchor, and if so, we then save this as our planeAnchor, which will be used as the base where to place the 3D model.

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
	
    guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
	
    if self.planeAnchor == nil {
        self.planeAnchor = planeAnchor
        self.loadDragonScene(with: planeAnchor)
    }
}

3D Models in SceneKit

ARKit integrates well with SpriteKit and SceneKit, Apple's respective 2D and 3D frameworks, which have been available for macOS and iOS for a number of years. Due to these years of development, Apple already has mature platforms which can be quickly hooked into an AR project to add 2D or 3D virtual elements.

There's a wide variety of 3D model formats available, but for this project, we are working with COLLADA (.dae) files. COLLADA is an open 3D format which many 3D modeling apps support. It was originally intended as an interchange format between competing 3D standards, but it has gained the support of a number of software tools, game engines and applications. COLLADA is also well supported in the Apple ecosystem, including the macOS Finder, Preview, and Xcode.

If your model has image textures which are referenced in the model file, then copy the .dae file and its associated image assets into the art.scnassets folder. One of the advantages of COLLADA being an open XML format is that the model file can be opened and edited with a standard text editor, which can be particularly useful if the image paths were improperly referenced (absolute path versus a relative path).

let dragonScene = SCNScene(named: "art.scnassets/Dragon_Baked_Actions_fbx_6.dae")!
let position = anchor.transform

// Iterate through all of the nodes and add them to the dragonNode object
for childNode in dragonScene.rootNode.childNodes {
    self.dragonNode.addChildNode(childNode)
}

// Scale and position the node
let scale:Float = 0.01
self.dragonNode.scale = SCNVector3(x: scale, y: scale, z: scale)
self.dragonNode.position = SCNVector3(x: position.columns.3.x, y: position.columns.3.y, z: position.columns.3.z)

// Add the dragonNode to the scene
sceneView.scene.rootNode.addChildNode(self.dragonNode)
self.dragonNode.isPaused = true // Initially pause any animations

Clearing Out Old Scenes

Loading in 3D models and the associated textures can be extremely memory intensive, so it is essential that any unused resources are properly released.

When removing a child node from a scene, it is not good enough to just call the node's removeFromParentNode() method. Any material objects from the node's geometry also need to be set to nil before removing the node from it's parent.

func clearScene() {

    sceneView.session.pause()
    sceneView.scene.rootNode.enumerateChildNodes { (node, stop) in
        // Free up memory when removing the node by removing any textures
        node.geometry?.firstMaterial?.normal.contents = nil
        node.geometry?.firstMaterial?.diffuse.contents = nil
        node.removeFromParentNode()
    }
}

Hit Detection

Being able to add objects to a scene is a key element for creating an augmented experience, but it does not provide much usefulness if one cannot interact with the environment. For this demonstration, tapping on the dragon will toggle its animation.

Upon tapping the screen, the sceneView will perform a hit test by extending a ray from where the screen was touched and will return an array of all of the objects which intersected the ray. The first object in the array is selected, which represents the object closest to the camera.

Since a 3D object might be comprised of multiple smaller nodes, the selected node might be a child node of a larger object. To check if the dragon model was tapped, the selected node's parent node is compared against the dragon node. If so, this will then call a method to toggle the model's animation.

func registerTapRecognizer() {
    let tapGestureRecognizer =  UITapGestureRecognizer (target:self ,action : #selector (screenTapped))
    self.sceneView.addGestureRecognizer(tapGestureRecognizer)
}

@objc func screenTapped(tapRecognizer: UITapGestureRecognizer) {
	
    let tappedLocation = tapRecognizer.location(in: self.sceneView)
    let hitResults = self.sceneView.hitTest(tappedLocation, options: [:])
    
    if hitResults.count > 0 {
        guard let firstHitResult = hitResults.first else {
            return
        }
        
        if self.dragonNode == firstHitResult.node.parent {
            self.toggleDragonAnimation()
        }
    }
}

Animations

Not all 3D models are static entities and some include animation effects. There are a variety of ways to start and stop animations, whether it is for a particular object or for the entire scene.

To toggle all animations for the scene requires just a single line of code:

self.sceneView.scene.isPaused = !self.sceneView.scene.isPaused

Toggling the animations for just a single node has similar functionality:

self.dragonNode.isPaused = !self.dragonNode.isPaused

These are simple methods to toggle the overall animation, but if you need more fine-grained control of the animations, then you will need to iterate through your SCNNode and modify each of the embedded animations.

Limitations

ARKit is just getting started, so there is still plenty of low-hanging fruit available to pick to improve the AR experience. Some of the current limitations are areas which can be improved upon with succeeding iterations of future hardware and software, but some issues will likely remain to be complex problems with no perfect solutions.

• No vertical plane detection : With the initial release of ARKit, it only supports the detection of horizontal planes (such as a floor or table top). Hopefully vertical plane detection will be supported in a future version, so walls can also be detected. (Update: This feature was added in ARKit 1.5.)
• Not particularly accurate : Placing virtual objects into the world is not a precise science, so the placement may not be entirely exact, or the objects may move a little bit or unexpectedly fly out of your view.
• Slow in detecting planes : Initial plane detection can take several seconds, which does not provide for an optimal user experience. If the lighting is poor or there are not enough distinct feature points available, plane detection takes even longer.
• Lack of distinct feature points makes plane detection difficult : If a plane does not contain enough distinct features (such as a blank, white surface), it will be difficult for ARKit to detect a suitable plane.
• Not effective for far distances : The phone and ARKit can estimate the distance to objects which are relatively close, but are poor in estimating objects which are far away (such as buildings or mountains).

Conclusion

Augmented Reality is in its nascent stages of development, which will provide many new and interesting ways for us to be able to use our mobile devices to interact with the world, whether it is for information, utility, or fun.

As the possibilities of what can be achieved with AR are explored further, more and more developers will delve into this new realm and see what they can create. Documentation and blog posts are invaluable in helping to reduce the initial learning curve and avoid some of the more troublesome hurtles that others have previously encountered, as this post aimed to accomplish by providing some tips on how to implement several common tasks when working with ARKit.

January 2018 Update

• I recently gave a presentation about ARKit at a Cocoahead's meeting. The Keynote slides can be downloaded here.
• This blog post is an original work which was started in October 2017 and was posted in December 2017, with minor updates made since then. If you come across a very similar blog post which uses the same screenshots, that is derived from this post which was credited to another person.

February 2018 Update

With the announcement of the upcoming ARKit 1.5 framework (currently available in the beta version of iOS 11.3) will be able to map to vertical surfaces, fixing one of the shortcomings in the original release of the framework.

December 2018 Update

This project has been updated to version 1.5, which now supports detection of vertical planes and adds the additional feature of being able to hang "portraits" on vertical surfaces. This corrects one of the major issues the initial version of ARKit had, but most of the other issues still remain. Plane detection of surfaces without distinctive features makes it difficult to properly detect surfaces.

For the first time in five years, Permanent Eraser has been updated on the Mac App Store. The major reason that Permanent Eraser has not been updated for the Mac App Store (MAS) for years is due to a rejection which occurred back with version 2.6.0, where the app was not allowed to add the Erase plug-in, which copies an Automator service to ~/Library/Services/. Since this is one of my favorite features of Permanent Eraser, I did not bother trying to update Permanent Eraser for MAS. The odd thing about this rejection, is that the version available on the store (version 2.5.3) had the same functionality which was the reason version 2.6.0 was rejected. Quite odd.

When macOS Sierra came out in 2016, this broke the MAS version of Permanent Eraser, since it relies upon the srm utility, which was no longer provided with Sierra. Permanent Eraser 2.6.0 and later contains its own custom version of srm, which fixes this problem. Since Permanent Eraser 2.5.3 was effectively rendered useless on the more modern versions of macOS, I decided to try and update it again for MAS, even if that required making a couple of sacrifices by limiting some of the functionality. A limited version of Permanent Eraser is better than a completely non-functional version. The other option would have been removing Permanent Eraser from MAS.

The first order of business was to determine what functionality needed to be removed to make Permanent Eraser compliant for the Mac App Store. Fortunately, there wasn't too much which needed to be excluded, primarily the plug-ins and the software update mechanism. Since MAS already offers its own capability to update the software, that was a non-issue. I'm hoping in a future incarnation of Permanent Eraser that I'll be able to directly integrate the plug-in service into the app, thus avoiding the issue of needing to manually installing the plug-in.

Once I had removed the necessary pieces from the app, that was only the beginning. Since the MAS version of Permanent Eraser hadn't been built for several years, it did not even initially compile, which required some tweaking of the project to get that to work again. Since this app is still built using Xcode 3 (so it can build as a Universal Binary for PowerPC and Intel), I needed to use the Application Loader app. However, one cannot just upload the app by itself. The app needs to be packaged first using the productbuild command line tool. In addition to the plethora of other Apple certificates I've generated for Mac and iOS apps, I also needed to generate a Mac Developer Installer certificate to properly sign and build the package. To package the app, I used the following command:

productbuild --component "Permanent Eraser.app" /Applications --sign "3rd Party Mac Developer Installer: Developer Name (12AB34567C)" \
--product "Permanent Eraser.app/Contents/Info.plist" "Permanent Eraser.pkg"

I was now able to upload the app, but quickly received an e-mail reporting a problem with the app:

Missing required icon - The application bundle does not contain an icon in ICNS format, containing both a 512x512 and a 512x512@2x image. For further assistance, see the Apple Human Interface Guidelines at https://developer.apple.com/macos/human-interface-guidelines/icons-and-images/app-icon

Since the current version of Permanent Eraser still supports Leopard, I normally do not include the 1024x1024 app icon, which causes problems in Leopard. But since all apps on MAS require Snow Leopard or later, this is not an issue. After doing some research, it appears that the best tools to generate the older .icns file is to use Icon Composer 2.2 (included with Xcode 4.3.3) or the command-line utility iconutil. If Icon Composer 2.2 is used, it must be used under Lion to generate the icon properly with the 1024x1024 image. I initially tried running Icon Composer 2.2 under Snow Leopard, but the generated icon did not include the 1024x1024 image. Running the app under Lion saved it properly.

Another alternative is to create an icon set folder with the required images and then convert it into an icon file. Add the following images to a new folder with the .iconset extension:

• appicon_16x16.png
• appicon_16x16@2x.png
• appicon_32x32.png
• appicon_32x32@2x.png
• appicon_128x128.png
• appicon_128x128@2x.png
• appicon_256x256.png
• appicon_256x256@2x.png
• appicon_512x512.png
• appicon_512x512@2x.png

iconutil -c icns -o appicon.icns appicon.iconset

With the proper icon in place, I tried uploading again. The next round of errors was due to an "invalid signature". I found this odd since I had already signed the app, yet MAS was reporting an issue. After digging around further, it appeared that I had used the incorrect certificate to sign the MAS version of the app. Yet another grey hair due to code signing issues. I initially used the certificate which started with "Mac Developer:", which is similar to what I had originally used years ago. The correct certificate is the one which starts with ""3rd Party Mac Developer Application:".

After climbing over these obstacles, Permanent Eraser was finally approved and is now up-to-date on the Mac App Store. Despite some of these unexpected issues, it did result in a learning process where I discovered a couple new things during the journey.

References

• Permanent Eraser on the Mac App Store
• productbuild man page
• Building for the Mac App Store
• Why doesn't icon composer 2.4 support the 1024x1024 size icon any more?
• Understanding the Code Signature
• Technical Note TN2206 - macOS Code Signing In Depth
• SignatureCheck Utility (Requires Apple Dev account)

Ydych chi'n siarad Cymraeg? Permanent Eraser does! Ar ben hynny mae'r app ar gael yn Gymraeg rwan! (The app is also available in Welsh now!)

Permanent Eraser 2.7.2, released on 17 November 2017, features a new Welsh localization and an updated Traditional Chinese localization. Many thanks to Applingua and Fangzhou for their incredible translation work.

This version finally introduces a feature I've been wanting to add for a long time — the ability to erase free space. This is currently available as the new Automator Action Erase Free Space, which joins the two previous Automator actions, which have also been updated in Permanent Eraser 2.7.2.

The ability to erase free space from Disk Utility has been removed in more recent versions of macOS, so if you have a hard drive which you want to wipe the free space, using the Erase Free Space Automator action is a solution.

This is just the first step in erasing free space. I'm hoping to be able to be able to integrate it directly into Permanent Eraser in a future version.

The full version history of what's new in Permanent Eraser 2.7.2:

• Added a new Automator action to erase free space from a drive.
• Updated the Erase and Erash Trash Automator actions.
• Added Welsh localization.
• Updated localization for Traditional Chinese.
• Tested and verified for macOS High Sierra.

The popular writing app Ulysses recently moved to a subscription payment model and they extensively explained their reasoning for the transition. Switching to a subscription model is rarely a popular move, which tends to incite the masses to pull out their torches and pitchforks.

Max Seelemann, development lead for Ulysses, does an excellent job in outlining the reasons for moving over to subscription in an attempt to assuage those people who bothered to read the article before they start angrily gnashing their teeth. One thing a company does not want to do when changing their pricing model (whether it be going from paid to subscription or paid to freemium) is to anger the current customers with this change.

Ideally, those early customers should be rewarded, and they should be enticed to remain loyal customers. Punishing them with a subscription fee on top of what they had previously paid for will not win many adoring fans. If you are changing your pricing model for your app, you should check if this was a customer before the change.

So how do you determine if someone had used an earlier version of an iOS app? We'll take a look at several different approaches below.

Check the build number in the receipt

Every app downloaded from the iOS or Mac App Store has a receipt, which contains a variety of information such as the bundle identifier, any in-app purchases, the subscription expiration date, the original app version, and more. The item we are interested in is the original app version. For iOS, this is the build number (CFBundleVersion), which is not the version number of the app (e.g. 2.1.3). On macOS, the receipt returns the CFBundleShortVersionString, instead, which is the version number of the app. If the build number for each release of your iOS app is unique, this is a valid solution to determine when a user first purchased the app. However, if the build numbers are not unique (such as the build numbers are reset back to 1 after each new release), then this can become quite problematic. On macOS, the receipt will contain the app's version number (e.g. 2.1.3), which is far more straightforward in determining which version of the app was initially purchased.

As long as build number scheme has been consistent and continues to increase and each version is unique, then this is a reliable method to compare the check if the user had an early version of the app before the pricing switch.

Check by the earliest date in any IAP

If your app offers in-app purchases (IAP), iterate through them and search for the oldest item. However, if your app does not offer IAP, this is not a viable option.

On-line accounts

If available in the app, if there is a login process and proper records stored on a server, when the user logs in to the app, the server could return information on when the user first registered. This would be dependent upon if the login process had recorded such information and could return it, as well. There is also the consideration of how an account was first created, whether it was through the iOS app, a website, or via some other method (another app, in-store sign up, etc.). This option is geared more towards a larger company which may have a larger resources and can afford to have multiple channels. This may not apply to all apps.

Roll your own method

These first three approaches can work if the app is reinstalled or downloaded onto another device. If the user has been using the app on the same device for awhile, the next several approaches can be used.

The app can save previous versions of the app to the device (either to the user defaults or a database), which can be checked against the current version of the app. Unfortunately, this will not work so well if the user has deleted the app (which deletes the user data, as well) and then reinstalls the app. Same applies if the app was installed on a new device, and the old local data may not exist or get transferred.

One method to be able to check if the app had been installed on that particular device at one time is to save some data in the Keychain, instead of the user defaults, which will persist even if the original app has been deleted.

Check creation date of the app's documentation folder

A clever approach to determine when the app was installed on the particular device is to check the creation date for the app's documents folder. The same problems arises, though, if the app has been newly installed or was deleted and then reinstalled on the device. Otherwise, this is a fairly reliable method to determine when the app was installed, but it should be used as the final check to determine how long the user has been using this app.

Objective-C:

NSError *error = nil;
NSURL *documentsFolderURL = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject];
NSDate *installDate = [[[NSFileManager defaultManager] attributesOfItemAtPath:documentsFolderURL.path error:&error] objectForKey:NSFileCreationDate];

if (error != nil) {
	NSLog(@"Error retrieving install date: %@", [error localizedDescription]);
}

Swift 3:

let documentsFolderURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).last!
let installDate = (try! FileManager.default.attributesOfItem(atPath: documentsFolderURL.path)[FileAttributeKey.creationDate])

January 2019 Update

In the WWDC 2018 video Best Practices and What’s New with In-App Purchases, they mentioned inspecting the Type 19 (original_application_version) attribute in your app's receipt to obtain the original application version.

From the WWDC session (emphasis mine):

So if you're paid up front and you want to move to a subscription or if you're paid up, paid up front and you want to move to a free trial use type 19. This will tell you the original version that the user purchased that app with from the App Store. Even if they delete the app and redownload it over and over again that type 19 will tell you exactly what version they bought their app with. If the user did originally pay up front make sure to give them the functionality that they bought. Just because you move to a subscription model if they bought it when it was a paid for app in the App Store you need to give them that access to that content they originally paid for.

So again use type 19 within the receipt to know what version they bought that app with.

As mentioned earlier in this post, the type 19 value corresponds to the CFBundleVersion in an iOS app, which is the build number and not the actual version number of the app. As long as the build version is unique for each released version of the app, this is a viable solution to determine if the user has used an earlier version of the app. However, if the build number is reset back to 1 after each new version of the app, this can cause major (and unintended) problems since the build numbers may not be unique or continually increase with each successive build of the app.

The inspiration for this post originated from a problem I encountered when I was working on an iOS app for a client who unwisely decided to switch from a paid app to a subscription model, which locked out all of their previous customers until they submitted to paying for the subscription. Upon encountering the uproar of a mob of very angry customers, this company decided to back pedal a bit and allow their current customers to still access their previously paid for content without having to pay for the subscription.

When I took on this project I discovered that the previous developer who had implemented the subscription feature had reset the build number for the app back to 1, which made it difficult and confusing to determine if a customer was a grandfathered user or not. This resulted in finding a number of alternative solutions to work around this problem which have been detailed in this post. Had the client put the appropriate amount of forethought into their poor business decision, they would have allowed their current customers to keep what they had already purchased without trying to burden them with an unwanted subscription. As Apple explicitly mentioned in this session, give them that access to that content they originally paid for.

If you are a company considering going down the subscription route, tread extremely careful, ere you might anger your entire customer base with a single misguided stroke. There may be no perfect or gentle way to switch to a subscription model, but there is certainly a very wrong way to go about it by taking away what your customers have already paid for. Until Apple and Google implement a method to provide for paid upgrades, developers may be forced into the subscription route to ensure that they have a more consistent stream of income versus a one-time payment for the entire lifespan of the app.

References

• Adding In-App Purchase to Your Applications
• (iOS + StoreKit) Can I detect when I'm in the sandbox?
• Trouble getting the original app version that the user installed (receipt validation)?
• iOS7 receipt contains CFBundleVersion instead of CFBundleShortVersionString
• Convert existing iOS paid app to freemium model with in-app purchase
• How to test app store purchase receipt to obtain original application version
• How to determine the date an app is installed or used for the first time?
• Determine if previously installed app
• Using RMStore library to get originalAppVersion
• Detect previous paid version of app
• Changing paid app to freemium (existing users)
• Ulysses Switches to Subscription
• Why we’re switching Ulysses to Subscription
• Using Store Kit for In-App Purchases with Swift 3
• Preventing Unauthorized Purchases with Receipts
• Using Receipts to Protect Your Digital Sales
• Best Practices and What’s New with In-App Purchases
• Receipt Validation Programming Guide
• Receipt Fields

During Apple's Platform State of the Union keynote, Apple revealed that macOS High Sierra would be the "last macOS release to support 32-bit apps without compromises." Considering that Apple has been warning us for a good year or more that 32-bit apps were going to be dropped in iOS, this does not come as too much of a surprise. I've had a few people inquire about a 64-bit version of 33 RPM being created. Even though 33 RPM has not been updated in nearly five years, this seemed like a simple enough request. Another Edenwaith app, Permanent Eraser, is an older app which is built for both 32 and 64 bit support, so I was hoping that all I would need to do is adjust a couple of settings and recompile 33 RPM.

Or so I hoped.

Returning to an old project was a good opportunity to perform some clean up and rethink some things with this app. I did not want to spend too much time on what was intended to be a minor update, otherwise I might tumble deep down the rabbit hole of endless features. A couple of ideas did seem plausible such as establishing a new code repository, code signing the app, fixing a few bugs which have crept up in newer versions of macOS, add TouchBar support, and compile the app with 64-bit support.

Considering that 33 RPM began its development in earnest in 2005, it was originally developed with a much earlier version of Xcode. The app was last updated in 2012, which still used Xcode 3 so it could be compiled as a Universal Binary for both PowerPC and Intel processors. Since most of the new ideas for the next version would be mostly keyed for more modern systems, I looked into compiling the app using Xcode 9 on macOS Sierra. Despite a 6 version difference between the original project and the current version of Xcode, it didn't act up too much upon trying to get the project to compile (in comparison to an old Project Builder project which would require finding an old version of Xcode to migrate it to the newer Xcode project format). Once I tried to compile the code, I first encountered the following error:

"QTKit/QTKit.h" file not found

Well.....huh. That's odd. 33 RPM is highly dependent upon QuickTime for a majority of its functionality, so why wouldn't the QTKit framework be found? The QTKit documentation page answers that question:

QuickTime Kit was deprecated in OS X v10.9. Use the AVFoundation framework instead.

Not only had QTKit been deprecated, but the framework was removed from Xcode 8. Fortunately, there is a workaround by extracting the MacOSX10.11.sdk from Xcode 7 and placing it in Xcode 8 or 9's Contents/Developer/Platforms/MacOSX.platform/Developer/SDKS/ folder.

After getting the MacOSX10.11.sdk put in place, it resolved the issue where the QTKit framework was not found, but it resulted in a barrage of 68 new errors, where many of the common QuickTime types (Movie, QTPropertyValueType, kQTMetaDataItemPropertyID_Key, etc.) could not be identified. When I went back to Xcode 3 and tried to compile the app with 64-bit support, it resulted in the same type of errors, so this seemed to indicate an issue with the 64-bit support. If I reverted the project back to 32-bit, it compiled without any issues.

Even with the older SDK in place, an app relying on QuickTime was not going to compile with 64-bit support. Apple has a Transitioning QTKit Code to AV Foundation guide, which recommends using the nm (display name list - symbol table) command line utility to get a list of the QuickTime APIs used within an app.

$ nm -m /Applications/33\ RPM.app/Contents/MacOS/33\ RPM | egrep "QuickTime|QTKit"
         (undefined [lazy bound]) external .objc_class_name_QTMovie (from QTKit)
         (undefined [lazy bound]) external .objc_class_name_QTMovieView (from QTKit)
         (undefined [lazy bound]) external _AttachMovieToCurrentThread (from QuickTime)
         (undefined [lazy bound]) external _DetachMovieFromCurrentThread (from QuickTime)
         (undefined [lazy bound]) external _DisposeMovie (from QuickTime)
         (undefined [lazy bound]) external _EnterMoviesOnThread (from QuickTime)
         (undefined [lazy bound]) external _ExitMoviesOnThread (from QuickTime)
         (undefined [lazy bound]) external _GetMediaSampleDescription (from QuickTime)
         (undefined [lazy bound]) external _GetMovieIndTrackType (from QuickTime)
         (undefined [lazy bound]) external _GetMovieTimeBase (from QuickTime)
         (undefined [lazy bound]) external _GetMovieTimeScale (from QuickTime)
         (undefined [lazy bound]) external _GetMovieTrackCount (from QuickTime)
         (undefined [lazy bound]) external _GetTrackDuration (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionBegin (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionEnd (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionFillBuffer (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionSetProperty (from QuickTime)
         (undefined [lazy bound]) external _MovieExportDoUserDialog (from QuickTime)
         (undefined [lazy bound]) external _MovieExportGetSettingsAsAtomContainer (from QuickTime)
         (undefined [lazy bound]) external _NewMovieFromHandle (from QuickTime)
         (undefined [lazy bound]) external _PutMovieIntoHandle (from QuickTime)
         (undefined [lazy bound]) external _QTCopyMovieMetaData (from QuickTime)
         (undefined [lazy bound]) external _QTGetComponentProperty (from QuickTime)
         (undefined [lazy bound]) external _QTGetComponentPropertyInfo (from QuickTime)
         (undefined [lazy bound]) external _QTGetMovieProperty (from QuickTime)
         (undefined [lazy bound]) external _QTGetMoviePropertyInfo (from QuickTime)
         (undefined [lazy bound]) external _QTGetTimeInterval (from QTKit)
         (undefined [lazy bound]) external _QTMakeTime (from QTKit)
         (undefined [lazy bound]) external _QTMakeTimeRange (from QTKit)
         (undefined) external _QTMediaTypeMPEG (from QTKit)
         (undefined) external _QTMediaTypeVideo (from QTKit)
         (undefined [lazy bound]) external _QTMetaDataGetItemCountWithKey (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataGetItemProperty (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataGetItemPropertyInfo (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataGetNextItem (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataRelease (from QuickTime)
         (undefined) external _QTMovieDataSizeAttribute (from QTKit)
         (undefined) external _QTMovieDidEndNotification (from QTKit)
         (undefined) external _QTMovieDisplayNameAttribute (from QTKit)
         (undefined) external _QTMovieEditableAttribute (from QTKit)
         (undefined) external _QTMovieExport (from QTKit)
         (undefined) external _QTMovieExportManufacturer (from QTKit)
         (undefined) external _QTMovieExportSettings (from QTKit)
         (undefined) external _QTMovieExportType (from QTKit)
         (undefined) external _QTMovieFileNameAttribute (from QTKit)
         (undefined) external _QTMovieLoadStateAttribute (from QTKit)
         (undefined) external _QTMovieLoopsAttribute (from QTKit)
         (undefined) external _QTMovieOpenAsyncOKAttribute (from QTKit)
         (undefined) external _QTMoviePlaysSelectionOnlyAttribute (from QTKit)
         (undefined) external _QTMovieRateAttribute (from QTKit)
         (undefined) external _QTMovieTimeScaleAttribute (from QTKit)
         (undefined) external _QTMovieURLAttribute (from QTKit)
         (undefined) external _QTMovieVolumeAttribute (from QTKit)
         (undefined [lazy bound]) external _QTSetComponentProperty (from QuickTime)
         (undefined [lazy bound]) external _QTSetMovieProperty (from QuickTime)
         (undefined [lazy bound]) external _QTSoundDescriptionGetProperty (from QuickTime)
         (undefined [lazy bound]) external _SCRequestImageSettings (from QuickTime)
         (undefined [lazy bound]) external .objc_class_name_QTMovie (from QTKit)
         (undefined [lazy bound]) external .objc_class_name_QTMovieView (from QTKit)
         (undefined [lazy bound]) external _AttachMovieToCurrentThread (from QuickTime)
         (undefined [lazy bound]) external _DetachMovieFromCurrentThread (from QuickTime)
         (undefined [lazy bound]) external _DisposeMovie (from QuickTime)
         (undefined [lazy bound]) external _EnterMoviesOnThread (from QuickTime)
         (undefined [lazy bound]) external _ExitMoviesOnThread (from QuickTime)
         (undefined [lazy bound]) external _GetMediaSampleDescription (from QuickTime)
         (undefined [lazy bound]) external _GetMovieIndTrackType (from QuickTime)
         (undefined [lazy bound]) external _GetMovieTimeBase (from QuickTime)
         (undefined [lazy bound]) external _GetMovieTimeScale (from QuickTime)
         (undefined [lazy bound]) external _GetMovieTrackCount (from QuickTime)
         (undefined [lazy bound]) external _GetTrackDuration (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionBegin (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionEnd (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionFillBuffer (from QuickTime)
         (undefined [lazy bound]) external _MovieAudioExtractionSetProperty (from QuickTime)
         (undefined [lazy bound]) external _MovieExportDoUserDialog (from QuickTime)
         (undefined [lazy bound]) external _MovieExportGetSettingsAsAtomContainer (from QuickTime)
         (undefined [lazy bound]) external _NewMovieFromHandle (from QuickTime)
         (undefined [lazy bound]) external _PutMovieIntoHandle (from QuickTime)
         (undefined [lazy bound]) external _QTCopyMovieMetaData (from QuickTime)
         (undefined [lazy bound]) external _QTGetComponentProperty (from QuickTime)
         (undefined [lazy bound]) external _QTGetComponentPropertyInfo (from QuickTime)
         (undefined [lazy bound]) external _QTGetMovieProperty (from QuickTime)
         (undefined [lazy bound]) external _QTGetMoviePropertyInfo (from QuickTime)
         (undefined [lazy bound]) external _QTGetTimeInterval (from QTKit)
         (undefined [lazy bound]) external _QTMakeTime (from QTKit)
         (undefined [lazy bound]) external _QTMakeTimeRange (from QTKit)
         (undefined) external _QTMediaTypeMPEG (from QTKit)
         (undefined) external _QTMediaTypeVideo (from QTKit)
         (undefined [lazy bound]) external _QTMetaDataGetItemCountWithKey (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataGetItemProperty (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataGetItemPropertyInfo (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataGetNextItem (from QuickTime)
         (undefined [lazy bound]) external _QTMetaDataRelease (from QuickTime)
         (undefined) external _QTMovieDataSizeAttribute (from QTKit)
         (undefined) external _QTMovieDidEndNotification (from QTKit)
         (undefined) external _QTMovieDisplayNameAttribute (from QTKit)
         (undefined) external _QTMovieEditableAttribute (from QTKit)
         (undefined) external _QTMovieExport (from QTKit)
         (undefined) external _QTMovieExportManufacturer (from QTKit)
         (undefined) external _QTMovieExportSettings (from QTKit)
         (undefined) external _QTMovieExportType (from QTKit)
         (undefined) external _QTMovieFileNameAttribute (from QTKit)
         (undefined) external _QTMovieLoadStateAttribute (from QTKit)
         (undefined) external _QTMovieLoopsAttribute (from QTKit)
         (undefined) external _QTMovieOpenAsyncOKAttribute (from QTKit)
         (undefined) external _QTMoviePlaysSelectionOnlyAttribute (from QTKit)
         (undefined) external _QTMovieRateAttribute (from QTKit)
         (undefined) external _QTMovieTimeScaleAttribute (from QTKit)
         (undefined) external _QTMovieURLAttribute (from QTKit)
         (undefined) external _QTMovieVolumeAttribute (from QTKit)
         (undefined [lazy bound]) external _QTSetComponentProperty (from QuickTime)
         (undefined [lazy bound]) external _QTSetMovieProperty (from QuickTime)
         (undefined [lazy bound]) external _QTSoundDescriptionGetProperty (from QuickTime)
         (undefined [lazy bound]) external _SCRequestImageSettings (from QuickTime)

With these problems in mind, where does this leave 33 RPM? With the deprecation of QuickTime and its lack of proper 64-bit support, this makes it impossible for 33 RPM to be a 64-bit app in its current state. The most obvious path is to transition the app from using QTKit to AV Foundation. If I do take this approach, it would also open up the possibility of also porting this app to iOS, which is something I've considered over the years, especially considering how abysmal, unfriendly, and ugly the stock iOS Music app has become over the past few years. This approach would likely take a fair amount of time, effectively rewriting the entire current app. Fortunately, we have at least another year before Apple really starts enforcing their new 64-bit only dictum. Apple also said, "In the next major release after High Sierra, we're going to aggressively start warning users if apps are not compatible for 64-bit.", so 32-bit apps may or may not work properly in 2018 and beyond, or we might see the coaxing messages we have seen on iOS for the past year or two which has warned users that older 32-bit iOS apps will not be supported in the future.

Had QuickTime not been deprecated and eschewed proper 64-bit support, I would have gladly made an update to 33 RPM so it would run on a future macOS without any additional compromises. However, it will likely take a good amount of time and effort to update 33 RPM to use the more modern AV Foundation framework. I will continue to look into this and determine how difficult it will actually be to update this app for more modern Apple platforms. But for now, continue to enjoy using 33 RPM and thanks to all who have supported and used 33 RPM over the past nine years.

References

• Platforms State of the Union 2017
• Apple to Phase Out 32-Bit Mac Apps Starting in January 2018
• macOS: How to See Which Mac Apps are 32-Bit
• QTKit documentation
• MacOS 10.12 Sierra/XCode 8 - Quicktime deprecation workaround
• Transitioning QTKit Code to AV Foundation
• nm man page

Several years ago I was doing some Android development. One of the things I appreciated the most about Android was the relative simplicity of being able to install apps to a device, unlike iOS which requires jumping through numerous hoops (although it is getting better with Xcode 9).

As a refresher for myself (and good documentation in the case I need to repeat these steps sometime in the future), here are the steps to take to install an APK app bundle onto an Android device.

1. Install the platform tools (such as adb). Download the platform tools from Google's website.
2. Unzip the downloaded file.
3. On the Mac, copy any wanted utilities to the /usr/local/bin folder (create the folder if necessary). For this purpose, adb needs to be copied over.
4. On an Android device, make sure the Developer mode is enabled. If not, go to Settings > About Phone/Tablet, then tap on the Build Number seven times to enable the Developer mode (otherwise, adb won't be able to see the device).
5. Verify that the device is visible with the command: adb devices
6. To install the apk file onto the device: adb install -r path/to/apk_file
7. If needed, uninstall the app with the command: adb uninstall com.example.appname

References

• SDK Platform Tools Release Notes
• SDK Platform-Tools for Mac
• Android Debug Bridge (adb)
• Connect Mobile Device with Android Debug Bridge(ADB) to USB, WiFi



Each year I play through at least one of the five Quest For Glory computer games, and this year it was time for Quest For Glory V. Whereas the first four games in the series can be played under DOSBox, QFG5 was released in 1998, which put it in the era of Windows 9X and the "Classic" era of Mac OS. Due to the hardware and software changes made in the past 19 years, QFG5 cannot be played natively on modern Macs. I've finally setup Sheepshaver on my main Mac, however I haven't experimented with it much, but it might be worth testing further in the future to determine if it can be a reliable replacement for aging hardware. Instead of using an emulator, I pulled out my Gigabit PowerMac G4, which can run every version of Mac OS from 9.0.4 through 10.4.11. This amazing machine was my main workhorse for seven years before I moved over to a new iMac. Sadly, time is starting to catch up with the G4, and I encountered a bevy of crashes, related both to dying hardware and the unreliability of a 90s-era operating system.

Using a computing system of yesteryear invokes sweet nostalgia, but that quickly wears off when one is rudely reminded of the shortcomings which have long been remedied with modern systems. Mac OS 9 lacked protected memory and preemptive multitasking, so one errant program could take down the entire system. Between the game freezing, the computer locking up, an entire boot volume gone AWOL and other hardware issues, completing QFG5 became much more difficult than normal by trying to fix and maintain the PowerMac.

After the first of several frustrating crashes, the main Macintosh HD volume refused to mount. After performing a scan with Disk Utility, it reported the following error:

invalid key length 4, 1579

sudo bless -folder9 "/Volumes/Macintosh HD/System Folder" --bootBlockFile "/usr/share/misc/bootblockdata"

Note: The -folder9 option has been removed from more modern versions of the bless utility, since Mac OS 9 has not been available on Macs since 2003.

This appeared to resolve the issues for a short while, until the computer crashed again, but this time, trying to fix the unmountable partition wasn't working. After digging further into old tricks on restoring Mac OS 9, I finally came across why the system was not booting properly any longer. When DiskWarrior tried rebuilding the Macintosh HD the second time, not all of the files were found. Another approach to bless the drive was to remove the System and Finder files from the System Folder, and then replace them. If everything goes well, the System Folder will get badged with a little Mac or Finder icon. What I encountered was that Finder file was missing! That would explain a lot why the system was not booting if a very critical part to the OS was missing.

The solution to fix this issue was to carefully reinstall Mac OS 9, and hopefully not have to wipe everything first. Since the Mac OS installer detected that a newer version of the OS already existed, it refused to install. By removing the existing System Folder, this helped convince the installer to begin installing. I started by installing Mac OS 9.1, which came along with my copy of Mac OS X 10.0. I then used the 9.2.1 upgrade disc that came with Mac OS X Jaguar for the next step. Trying to find the 9.2.2 installer proved to be a little more difficult and required a little digging across the internet to find a working version (albeit, on a very sluggish FTP server). A copy of the Mac OS 9.2.2 upgrade installer is available here.

Fortunately, even after telling the Mac OS installer to perform a clean install, it left most of the original files intact, so I did not have to reinstall much except for the driver for the ATI Radeon 9000 video card.

Things looked good again! Well, for a few minutes, at least. I started getting odd glitches and errors again, such as when I tried to launch QFG5, the OS claimed it could not find the DrawSprocketLib, but when I tried again, the game launched fine. Then the game started to get extremely sluggish, and eventually froze. When I restarted the computer, the Macintosh HD could not be found as a bootable drive again. (sigh)

The number of errors I was encountering indicated something far past software-related issues, so I started performing more extensive hardware tests using DiskWarrior, Data Rescue II, and the Apple Hardware Test disc that came with the computer. The issues I kept experiencing seemed to indicate some bad sectors on the hard drive, but any of the tools did not report the disk being faulty. The extended Apple hardware test did reveal that one of the sticks of RAM was bad (error: mem_/2/4), which is a likely culprit to random errors, lock ups, and crashes. I removed the bad stick of RAM (and fortunately the RAM sticks didn't need to be paired on this particular Mac), which was somewhat sad, since that was 512MB of RAM on a system which had 1.5GB of RAM, a very impressive amount for a machine of that age. Still, 1GB of RAM is more than enough for Mac OS 9. I'm hoping that the bad RAM is the source of all of the issues and that the hard drive isn't also dying, since trying to find a small hard drive would become increasingly difficult during this time where most storage has moved over to SSD. I might need to look into other options in the future to equip the G4 with an SSD.

I ended up completing QFG5 by playing in Classic mode from one of my Mac OS X partitions, which was passable, but not as smooth as playing natively in Mac OS 9. I finished it this way since the Mac OS 9 drive was available in the Classic system preference pane so I could make use of the System Folder to launch the Classic mode, but I still wasn't able to boot back into Mac OS 9. Disk Utility was once again reporting an invalid key length and the disk could not be repaired. Time for DiskWarrior, again. I tried DiskWarrior once more, but when it tried to rebuild the disk, it kept locking up at a certain spot (sigh again). Classic mode it would be, before digging into this problem once again.

Next up I made use of the command line utility fsck_hfs to attempt to verify and repair the hard drive.

[Ixia:~] % sudo fsck_hfs -r -d /dev/disk0s10
Password:
** /dev/rdisk0s10
** Checking HFS Plus volume.
** Checking Extents Overflow file.
** Checking Catalog file.
** Rebuilding Catalog B-tree.
hfs_UNswap_BTNode: invalid node height (1)
hfs_swap_HFSPlusBTInternalNode: catalog key #9 invalid length (8220)
   Invalid key length
(4, 1335)
hfs_swap_HFSPlusBTInternalNode: catalog key #32 invalid length (12338)
   Invalid key length
(4, 1614)
** Rechecking volume.
** Checking HFS Plus volume.
** Checking Extents Overflow file.
** Checking Catalog file.
   Missing thread record (id = 805355401)
   Invalid extent entry
(4, 1504)
   Incorrect block count for file Finder Preferences
   (It should be 0 instead of 12288)
   Invalid extent entry
(4, 1504)
   Incorrect block count for file Finder Preferences
   (It should be 1 instead of 8193)
   Invalid extent entry
(4, 1504)
   Incorrect block count for file FonၴAnnexFiぬe
   (It should be 0 instead of 1)
   Invalid extent entry
(4, 1504)
   Incorrect size for file FonၴAnnexFiぬe
   (It should be 0 instead of 536870912)
   Invalid extent entry
(4, 1504)
   Incorrect block count for file Game Sばrockets〠Update ⁐refs
   (It should be 12289 instead of 8193)
   Invalid extent entry
(4, 1504)
   Incorrect block count for file Game Sばrockets〠Update ⁐refs
   (It should be 8193 instead of 12289)
   Invalid extent entry
(4, 1504)
   Invalid extent entry
(4, 1581)
   Invalid extent entry
(4, 1581)
   Incorrect block count for file Standard Additions
   (It should be 12299 instead of 11)
   Incorrect size for file 4/C Ctd. TRUMATCH/RIၔ/Profil⁥80
   (It should be 909312 instead of 52776559040052)
   Incorrect number of thread records
(4, 260)
        CheckCatalogBTree: dirCount = 6565, dirThread = 6583
   Incorrect number of thread records
(4, 260)
        CheckCatalogBTree: fileCount = 34766, fileThread = 34718
** Checking Catalog hierarchy.
   Missing thread record (id = 805355401)
   Invalid directory item count
   (It should be 7 instead of 45)
   Missing thread record (id = 805359315)
   Invalid directory item count
   (It should be 0 instead of 3)
   Invalid volume directory count
   (It should be 6355 instead of 6564)
   Invalid volume file count
   (It should be 32838 instead of 34766)
** Checking Extended Attributes file.
   Incorrect number of Extended Attributes
(8, 1)
        extentType=0x0, startBlock=0x277437, blockCount=0x1, attrName=(null)
   Overlapped extent allocation (file 805355895)
        extentType=0x0, startBlock=0x277439, blockCount=0x6, attrName=(null)
   Overlapped extent allocation (file 805355897)
        extentType=0xff, startBlock=0x277441, blockCount=0x1, attrName=(null)
   Overlapped extent allocation (file 805355898)   
        extentType=0x0, startBlock=0x277442, blockCount=0x1, attrName=(null)
        
    ... (several thousand more lines like this)
      
   Overlapped extent allocation (file 805362187)
** Checking volume bitmap.
   Volume Bit Map needs minor repair
** Checking volume information.
   Invalid volume free block count
   (It should be 2729961 instead of 2797618)
        invalid VHB nextCatalogID 
   Volume Header needs minor repair
(2, 0)
   Verify Status: VIStat = 0xa800, ABTStat = 0x0040 EBTStat = 0x0000
                  CBTStat = 0x0800 CatStat = 0x4230
** Repairing volume.
   Cannot create links to all corrupt files
** The volume Macintosh HD could not be repaired.
        volume type is embedded HFS+ 
        primary MDB is at block 2 0x02 
        alternate MDB is at block 41680894 0x27bfffe 
        primary VHB is at block 3226 0xc9a 
        alternate VHB is at block 41680662 0x27bff16 
        sector size = 512 0x200 
        VolumeObject flags = 0x1F 
        total sectors for volume = 41680896 0x27c0000 
        total sectors for embedded volume = 41677440 0x27bf280 
[Ixia:~] % 

I tried to force rebuild the catalog with the command sudo fsck_hfs -y -r -d /dev/disk0s10, but it was the same error as what Disk Utility displayed that the disk could not be repaired. That was enough fun and hair pulling for that night.

The next day...the computer booted up fine into Mac OS 9! Well, go figure. That's computers. Sometimes it does pay to step away from the problem and come back later. Perhaps the hardware is still having issues and giving it a rest fixed the problem for now (shrug). Either way, it probably is time to either resolve any additional hardware issues or finally retire this trusty workhorse of a Mac.

Working and playing with Mac OS 9 for a couple of weeks was an interesting experience. Some tools such as Transmit 1.7, BBEdit Lite 6.1.2 and Classilla proved to be quite useful, but other tools needed to be dug up or experimented with. I also tried out USBOverdrive 1.4, but it did not seem to work with the Razer DeathAdder mouse, and the mouse cursor locked up, so I had to restart the computer and hold down Shift to disable the Extensions and then permanently disable USBOverdrive. In addition to the software previously mentioned, I even downloaded ResEdit for fun.

Special thanks to those sites like Macintosh Repository and Mac OS 9 Lives for providing resources to keep the "classic" Mac OS alive, which proved useful in searching for old software.

References

• invalid key length, disk repair fails
• Invalid Key Length error after 10.4.3 install/10.4.4 update
• Fix for Invalid key length - Volume check failed in Mac
• Mac Can’t Repair HFS Hard Drive. How to Resolve?
• Repair disk from single user mode (fsck fails)

As a set of interesting programming exercises, I've written up a number of examples to calculate the amount of free space on a given disk running on macOS. The first two examples are command line solutions, while the other examples take more programatic approaches.

Update (26 November 2017)

Due to some new approaches I came across from this StackOverflow post, two additional methods have been added by using diskutil and the private macOS framework DiskManagement.

df

To easily display the amount of available disk space and how much is free, use the built in utility df which returns the statistics about the amount of free disk space on a specified filesystem. This is a great little utility to use from the terminal or within a script. To get the details on the root drive, run the command df -kP / which will return data formatted like the following:

	Filesystem 1024-blocks      Used  Available Capacity  Mounted on
	/dev/disk2  2018614912 698624560 1319734352    35%    /

If you want just the amount of free disk space (calculated in kilobytes), use one of these two options which calls df and then parses out the pertinent data.

	df -kP / | tail -1 | awk '{print $4}'
	df -kP / | awk '/[0-9]%/{print $(NF-2)}'

diskutil

diskutil is a veritable utility for disk related information and maintenance tasks, which serves as the command line version of the Disk Utility app. To retrieve the amount of free space (sometimes referred to as "available space"), run the command. diskutil info / | grep "Available Space". Due to changes in the output from diskutil over the years, on older version of Mac OS X, use the command diskutil info / | grep "Free Space".

$ diskutil info / | grep "Available Space"
   Volume Available Space:   94.0 GB (93988098048 Bytes) (exactly 183570504 512-Byte-Units) (37.6%)

DiskManagement Framework

Ivan Genchev discovered an interesting approach to determine the available free space by using the private macOS framework DiskManagement, which is used by diskutil (and I assume, Disk Utility, as well). Since this framework is not public, you'll need to generate a header file by using Steve Nygard's excellent utility class-dump, which allows you to examine the Objective-C runtime information stored in a Mach-O file. This is the same approach I used in another project to write a Finder plug-in, which required generating a Finder.h header file to be able to access the private Finder methods.

Download class-dump and copy the executable onto your system, such as in your /usr/local/bin directory.

Next, generate the header file with the command:

 
$ class-dump /System/Library/PrivateFrameworks/DiskManagement.framework/Versions/Current/DiskManagement > DiskManagement.h

This will create the necessary header file to link to the program. This is a fairly large file (the one for macOS High Sierra is 840 lines in length), and contains a number of interesting private methods. The method we are interested in is (id)volumeFreeSpaceForDisk:(struct __DADisk *)arg1 error:(int *)arg2.

Once we have the DiskManagement.h file, we can integrate it into our program. The following code is modified from Ivan Genchev's code in the StackOverflow post.

/*	dmfreespace.m
 *
 *	Description: Get the available free space on the root drive using the method 
 *	volumeFreeSpaceForDisk from the private framework DiskManagement
 *	
 *	Original reference: https://stackoverflow.com/a/20679389/955122
 *	To compile: clang -g dmfreespace.m -F/System/Library/PrivateFrameworks/ -framework Foundation 
 *	-framework DiskArbitration -framework DiskManagement -o dmfreespace
 */

#import <Foundation/Foundation.h>
#import "DiskManagement.h"
#import <DiskArbitration/DADisk.h>
// For statfs
#include <sys/param.h>
#include <sys/mount.h>


int main(int argc, char *argv[])
{
    int                 err = 0;
    const char *        bsdName;
    DASessionRef        session;
    DADiskRef           disk;
    CFDictionaryRef     descDict;
    NSString *.         rootPath = @"/";
    
    session  = NULL;
    disk     = NULL;
    descDict = NULL;
    
    // Get the BSD name for the given path
    struct statfs devStats;
    statfs([rootPath UTF8String], &devStats);
    bsdName = devStats.f_mntfromname;
    NSLog(@"bsdName: %s", bsdName);
    
    if (err == 0) {session = DASessionCreate(NULL); if (session == NULL) {err = EINVAL;}}
    if (err == 0) {disk = DADiskCreateFromBSDName(NULL, session, bsdName); if (disk == NULL) {err = EINVAL;}}
    if (err == 0) {descDict = DADiskCopyDescription(disk); if (descDict == NULL) {err = EINVAL;}}

    DMManager *dmMan = [DMManager sharedManager];
    NSLog(@"blockSizeForDisk: %@", [dmMan blockSizeForDisk:disk error:nil]);
    NSLog(@"totalSizeForDisk: %@", [dmMan totalSizeForDisk:disk error:nil]);
    NSLog(@"volumeTotalSizeForDisk: %@", [dmMan volumeTotalSizeForDisk:disk error:nil]);
    NSLog(@"volumeFreeSpaceForDisk: %@", [dmMan volumeFreeSpaceForDisk:disk error:nil]);
    
    return 0;
}

Run the app and you should get output like the following:

 $ ./dmfreespace
2017-11-26 12:35:44.945 dmfreespace[17103:2781845] bsdName: /dev/disk1s1
2017-11-26 12:35:44.969 dmfreespace[17103:2781845] blockSizeForDisk: 4096
2017-11-26 12:35:44.970 dmfreespace[17103:2781845] totalSizeForDisk: 250035572736
2017-11-26 12:35:44.971 dmfreespace[17103:2781845] volumeTotalSizeForDisk: 250035572736
2017-11-26 12:35:44.971 dmfreespace[17103:2781845] volumeFreeSpaceForDisk: 93637120000

A GitHub Gist with dmfreespace.m and DiskManagement.h is available here.

Cocoa

The following program freesize.m takes five different approaches to calculate the free space on a drive. The first two examples use Cocoa's Foundation framework. The first example uses NSFileManager and asks for the NSFileSystemFreeSize attribute. Simple and straight forward.

The second example uses NSURL with some more modern approaches. This example gets a URL's resources and requests the value for the key NSURLVolumeAvailableCapacityKey. This code also makes use of NSByteCountFormatter, which was introduced with OS X 10.8 Mountain Lion. This method is a little more useful than the original way, since this can display the total available free space, which tends to be slightly smaller than all of the free space, not all which is available to the user, since some space might be reserved for the system.

getattrlist

	Volume size: 16229794380578291739
	Free space:  15103894473735667713
	Avail size:  1

This was obviously incorrect, which makes me suspect that the vol_attr structure was not lining up properly with the returned data, and I might even need to add some sort of padding inside the structure. There are far more easier methods to obtain the free space size than by using getattrlist.

statfs + statvfs

The final two examples are similar UNIX system calls to get file system information and statistics. These methods statfs and statvfs are nearly identical in how a structure is passed into a system call and then the necessary information is parsed out. The one major difference I found between these two system calls was that the f_bsize returned different values. The statfs structure returns a value of 4096 for f_bsize, whereas statvfs returns a value of 1048576, a value 256 times larger than the other one. To properly calculate the available free space using the statvfs call, I needed to use f_frsize instead. Multiply f_frsize by f_bavail and that will result in the total number of available bytes. To calculate that number in kilobytes, divide that product by 1024.

freesize.m Gist

Purgeable Space

In more current versions of macOS, if you pull up the Get Info window on a selected disk, the amount of available space might be different from the values which was calculated in the examples above. However, there is a second value which details how much space is purgeable. If you subtract the purgeable space from the available space, you should get a value very similar to the one calculated above. This purgeable space seems to come from snapshots taken by Time Machine or APFS, which are occasionally cleaned up (or purged).

References

• Mac OS X Internals
• Mac OS X 10.4 getattrlist Know File System Type
• permtype-getattrlist.m
• permtype.m
• File Ownership and ACLS
• Converting statvfs to percentage free correctly
• How to Get Available Filesystem Space on Linux: a C Function with a C++ Example
• differ in output of df and statfs
• statfs man page
• mountedVolumeURLsIncludingResourceValuesForKeys:options: reference page
• Cocoa apis reporting incorrect values for free space, what should I use?
• NSByteCountFormatter reference page
• How to get storage capacity of mac programatically?
• How to get diskutil info output in a cocoa application
• class-dump
• DiskManagement.h + dmfreespace.m Gist
• freesize.m Gist
• Mobile Time Machine and its transformation in High Sierra