android - How to get free and total size of each StorageVolume?

Question

Background

Google (sadly) plans to ruin storage permission so that apps won't be able to access the file system using the standard File API (and file-paths). Many are against it as it changes the way apps can access the storage and in many ways it's a restricted and limited API.

As a result, we will need to use SAF (storage access framework) entirely on some future Android version (on Android Q we can, at least temporarily, use a flag to use the normal storage permission), if we wish to deal with various storage volumes and reach all files there.

So, for example, suppose you want to make a file manager and show all the storage volumes of the device, and show for each of them how many total and free bytes there are. Such a thing seems very legitimate, but as I can't find a way to do such a thing.

The problem

Starting from API 24 (here), we finally have the ability to list all of the storage volumes, as such:

    val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
    val storageVolumes = storageManager.storageVolumes

Thing is, there is no function for each of the items on this list to get its size and free space.

However, somehow, Google's "Files by Google" app manages to get this information without any kind of permission being granted :

And this was tested on Galaxy Note 8 with Android 8. Not even the latest version of Android.

So this means there should be a way to get this information without any permission, even on Android 8.

What I've found

There is something similar to getting free-space, but I'm not sure if it's indeed that. It seems as such, though. Here's the code for it:

    val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
    val storageVolumes = storageManager.storageVolumes
    AsyncTask.execute {
        for (storageVolume in storageVolumes) {
            val uuid: UUID = storageVolume.uuid?.let { UUID.fromString(it) } ?: StorageManager.UUID_DEFAULT
            val allocatableBytes = storageManager.getAllocatableBytes(uuid)
            Log.d("AppLog", "allocatableBytes:${android.text.format.Formatter.formatShortFileSize(this,allocatableBytes)}")
        }
    }

However, I can't find something similar for getting the total space of each of the StorageVolume instances. Assuming I'm correct on this, I've requested it here.

You can find more of what I've found in the answer I wrote to this question, but currently it's all a mix of workarounds and things that aren't workarounds but work in some cases.

The questions

1. Is getAllocatableBytes indeed the way to get the free space?
2. How can I get the free and real total space (in some cases I got lower values for some reason) of each StorageVolume, without requesting any permission, just like on Google's app?

Answer 1

Is getAllocatableBytes indeed the way to get the free space?

Android 8.0 Features and APIs states that getAllocatableBytes(UUID):

Finally, when you need to allocate disk space for large files, consider using the new allocateBytes(FileDescriptor, long) API, which will automatically clear cached files belonging to other apps (as needed) to meet your request. When deciding if the device has enough disk space to hold your new data, call getAllocatableBytes(UUID) instead of using getUsableSpace(), since the former will consider any cached data that the system is willing to clear on your behalf.

So, getAllocatableBytes() reports how many bytes could be free for a new file by clearing cache for other apps but may not be currently free. This does not seem to be the right call for a general-purpose file utility.

In any case, getAllocatableBytes(UUID) doesn't seem to work for any volume other than the primary volume due to the inability to get acceptable UUIDs from StorageManager for storage volumes other than the primary volume. See Invalid UUID of storage gained from Android StorageManager? and Bug report #62982912. (Mentioned here for completeness; I realize that you already know about these.) The bug report is now over two years old with no resolution or hint at a work-around, so no love there.

If you want the type of free space reported by "Files by Google" or other file managers, then you will want to approach free space in a different way as explained below.

How can I get the free and real total space (in some cases I got lower values for some reason) of each StorageVolume, without requesting any permission, just like on Google's app?

Here is a procedure to get free and total space for available volumes:

Identify external directories: Use getExternalFilesDirs(null) to discover available external locations. What is returned is a File[]. These are directories that our app is permitted to use.

extDirs = {File2@9489
0 = {File@9509} "/storage/emulated/0/Android/data/com.example.storagevolumes/files"
1 = {File@9510} "/storage/14E4-120B/Android/data/com.example.storagevolumes/files"

(N.B. According to the documentation, this call returns what are considered to be stable devices such as SD cards. This does not return attached USB drives.)

Identify storage volumes: For each directory returned above, use StorageManager#getStorageVolume(File) to identify the storage volume that contains the directory. We don't need to identify the top-level directory to get the storage volume, just a file from the storage volume, so these directories will do.

Calculate total and used space: Determine the space on the storage volumes. The primary volume is treated differently from an SD card.

For the primary volume: Using StorageStatsManager#getTotalBytes(UUID get the nominal total bytes of storage on the primary device using StorageManager#UUID_DEFAULT . The value returned treats a kilobyte as 1,000 bytes (rather than 1,024) and a gigabyte as 1,000,000,000 bytes instead of 2³⁰. On my SamSung Galaxy S7 the value reported is 32,000,000,000 bytes. On my Pixel 3 emulator running API 29 with 16 MB of storage, the value reported is 16,000,000,000.

Here is the trick: If you want the numbers reported by "Files by Google", use 10³ for a kilobyte, 10⁶ for a megabyte and 10⁹ for a gigabyte. For other file managers 2¹⁰, 2²⁰ and 2³⁰ is what works. (This is demonstrated below.) See this for more information on these units.

To get free bytes, use StorageStatsManager#getFreeBytes(uuid). Used bytes is the difference between total bytes and free bytes.

For non-primary volumes: Space calculations for non-primary volumes is straightforward: For total space used File#getTotalSpace and File#getFreeSpace for the free space.

Here are a couple of screens shots that display volume stats. The first image shows the output of the StorageVolumeStats app (included below the images) and "Files by Google." The toggle button at the top of the top section switches the app between using 1,000 and 1,024 for kilobytes. As you can see, the figures agree. (This is a screen shot from a device running Oreo. I was unable to get the beta version of "Files by Google" loaded onto an Android Q emulator.)

The following image shows the StorageVolumeStats app at the top and output from "EZ File Explorer" on the bottom. Here 1,024 is used for kilobytes and the two apps agree on the total and free space available except for rounding.

MainActivity.kt

This small app is just the main activity. The manifest is generic, compileSdkVersion and targetSdkVersion are set to 29. minSdkVersion is 26.

class MainActivity : AppCompatActivity() {
    private lateinit var mStorageManager: StorageManager
    private val mStorageVolumesByExtDir = mutableListOf<VolumeStats>()
    private lateinit var mVolumeStats: TextView
    private lateinit var mUnitsToggle: ToggleButton
    private var mKbToggleValue = true
    private var kbToUse = KB
    private var mbToUse = MB
    private var gbToUse = GB

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        if (savedInstanceState != null) {
            mKbToggleValue = savedInstanceState.getBoolean("KbToggleValue", true)
            selectKbValue()
        }
        setContentView(statsLayout())

        mStorageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager

        getVolumeStats()
        showVolumeStats()
    }

    override fun onSaveInstanceState(outState: Bundle) {
        super.onSaveInstanceState(outState)
        outState.putBoolean("KbToggleValue", mKbToggleValue)
    }

    private fun getVolumeStats() {
        // We will get our volumes from the external files directory list. There will be one
        // entry per external volume.
        val extDirs = getExternalFilesDirs(null)

        mStorageVolumesByExtDir.clear()
        extDirs.forEach { file ->
            val storageVolume: StorageVolume? = mStorageManager.getStorageVolume(file)
            if (storageVolume == null) {
                Log.d(TAG, "Could not determinate StorageVolume for ${file.path}")
            } else {
                val totalSpace: Long
                val usedSpace: Long
                if (storageVolume.isPrimary) {
                    // Special processing for primary volume. "Total" should equal size advertised
                    // on retail packaging and we get that from StorageStatsManager. Total space
                    // from File will be lower than we want to show.
                    val uuid = StorageManager.UUID_DEFAULT
                    val storageStatsManager =
                        getSystemService(Context.STORAGE_STATS_SERVICE) as StorageStatsManager
                    // Total space is reported in round numbers. For example, storage on a
                    // SamSung Galaxy S7 with 32GB is reported here as 32_000_000_000. If
                    // true GB is needed, then this number needs to be adjusted. The constant
                    // "KB" also need to be changed to reflect KiB (1024).
//                    totalSpace = storageStatsManager.getTotalBytes(uuid)
                    totalSpace = (storageStatsManager.getTotalBytes(uuid) / 1_000_000_000) * gbToUse
                    usedSpace = totalSpace - storageStatsManager.getFreeBytes(uuid)
                } else {
                    // StorageStatsManager doesn't work for volumes other than the primary volume
                    // since the "UUID" available for non-primary volumes is not acceptable to
                    // StorageStatsManager. We must revert to File for non-primary volumes. These
                    // figures are the same as returned by statvfs().
                    totalSpace = file.totalSpace
                    usedSpace = totalSpace - file.freeSpace
                }
                mStorageVolumesByExtDir.add(
                    VolumeStats(storageVolume, totalSpace, usedSpace)
                )
            }
        }
    }

    private fun showVolumeStats() {
        val sb = StringBuilder()
        mStorageVolumesByExtDir.forEach { volumeStats ->
            val (usedToShift, usedSizeUnits) = getShiftUnits(volumeStats.mUsedSpace)
            val usedSpace = (100f * volumeStats.mUsedSpace / usedToShift).roundToLong() / 100f
            val (totalToShift, totalSizeUnits) = getShiftUnits(volumeStats.mTotalSpace)
            val totalSpace = (100f * volumeStats.mTotalSpace / totalToShift).roundToLong() / 100f
            val uuidToDisplay: String?
            val volumeDescription =
                if (volumeStats.mStorageVolume.isPrimary) {
                    uuidToDisplay = ""
                    PRIMARY_STORAGE_LABEL
                } else {
                    uuidToDisplay = " (${volumeStats.mStorageVolume.uuid})"
                    volumeStats.mStorageVolume.getDescription(this)
                }
            sb
                .appendln("$volumeDescription$uuidToDisplay")
                .appendln(" Used space: ${usedSpace.nice()} $usedSizeUnits")
                .appendln("Total space: ${totalSpace.nice()} $totalSizeUnits")
                .appendln("----------------")
        }
        mVolumeStats.text = sb.toString()
    }

    private fun getShiftUnits(x: Long): Pair<Long, String> {
        val usedSpaceUnits: String
        val shift =
            when {
                x < kbToUse -> {
                    usedSpaceUnits = "Bytes"; 1L
                }
                x < mbToUse -> {
                    usedSpaceUnits = "KB"; kbToUse
                }
                x &