Loading PQs from Mac to GPSmap 67i

[+TheWaynesInAZ]

Greetings--

 

Just upgraded to a GPSmap 67i from a 64st, added a 32gb microSD card, and the first thing I noticed it that when plugged into the Mac, it no longer shows up as a drive (or 2) on my desktop, thus my iCaching software doesn't see it directly for purposes of export. Garmin support told me I needed the Android File Transfer utility see the units directory and transfer files to the GPX directory. That worked, booted up and voila!. Tried to find them again the next day and nothing. Support had me verify the PQs were enabled and to use both AFT to load files to the directory, import them into BaseCamp then Xfer to the 67i. After reinstating BC (not my favorite piece of software), I was able to get the PQ loaded onto the unit. Not exactly a clean way to do so after the iCaching/GPSmap 64st combo I've used for years.

 

Does anyone know a more efficient way to do so? I can run Windows and try GSAK, but would rather not.

[+Mineral2]

huh... interesting that the 67 is built on android. 

[JaVaWa]

It's not. The GPSMAP 67 uses MTP (Media Transfer Protocol, designed by Microsoft) for communication with PC or Mac. MTP support by Mac OS is limited, Finder cannot access MTP devices. When Google stopped using USB mass storage and switched to MTP in Android, they made a tool to access Android devices in MacOS and called it "Android File Transfer". The proper name would be  "MTP File Transfer", since it isn't really Android related.

[+Mineral2]

ok... interesting that Garmin is giving up USB mass storage. That makes it much harder to transfer data back and forth.

[robertlipe]

It's just inevitable, @Mineral2. If you'll think about it, you've seen this progression for decades. FAT filesystems were of the CPM and later DOS era then things were coded in assembly and for a single CPU to be reading and writing them.

Imagine a filesystem on a "disk" (doesn't matter if it's a memory card or a shared buffer or punched paper or mercury tubes or whatever...) as a clay tablet (is that all I've got? We're going to overlook that a clay-tablet is write-mostly? Really?  I've gotta work on my metaphors... :-) ) that always has to be internally consistent with itself in case it's suddenly disconnected from the writer. It has separate areas like an index and a table of contents and it has a list of space of what space has already been written on and what's free. With some clever chisel-work, you can make sure that this always works by having the chisel that carves out letters also updating the TOC and the free list all in the same blow of the hammer. It's atomic (unbreakable) on each hammer strike. Sure, the chisel is funny looking, but this is how filesystems work. If the writer is stricken by a diety while they're writing, the tablet is always self-consistent and there's no chance of it getting out of sync.

Now, if you have TWO writers trying to hammer away on that same block, each with their own magic chisels, each may try to write into the same blank space twice, each with different data which will hose up the table of contents/index if block 1,347 could possibly contain two different entries.

The wheels pretty much fall off of everything if you have TWO processors trying to write to the same filesystem. You've seen evidence of this for decades. Network operating systems work very hard so that hundreds of computers talk to IT and IT talks to the filesystems, introducing locking and such, so prevent this problem. (Remember 3Com and Novel?)  Cameras (remember those?)  have long had this problem. If the camera is displaying the list of photos while you're connecting to a computer and adding, deleting, and reordering photos, Bad Things happen. We introduced PTP, the Picture Tunneling Protocol to act like an NOS.  The CPU on your MP3 Players (remember those?) can't index your songs and display your albums and play lists while you're connecting to a computer and reordering, adding, and removing things. So we extended PTP to become the Media Tunneling Protocol.  The final entry in my walk down history lane will indeed be cell phones, notably the very Android that's mentioned when it added memory cards, introducing Android File Transfer (which has applications outside of Android, but has the advantage of being open source and widely adopted by now as well as open implementations for all the OSes that matter) to be the intermediary where everything (the big computer with a keyboard and the tiny computer with a battery) spoke a protocol to AFT and AFT spoke to the tablet, err, storage media. In all these examples, these things handled notifying the other readers/writers when a change happened, when another device connected, and so on.

If you think about it, we've seen the same issues in GPSes for years.The Garmin 60CSx wouldn't let you store anything but maps on the SD card and it required a reboot to read them. The Garmin and the host couldn't both access the card at the same time. (Contrast that to the protocol-driven devices where you could watch them draw waypoints on the screen as they were added by software like mine as they transferred.) Nuvi 350, back in 2005, and before it, the i3, would basically go into a catatonic state with the local CPU doing nothing as long as the USB connection was detected. (This was annoying as hell if your charging cable happened to introduce Just Enough resistance on the pin it was supposed to leave unconnected so that your car charger would put your GPS into this flatline state.) When the USB cable disconnected, the device essentially rebooted, invalidating what it knew about the state of the clay tablet, err, filesystem, and would read them fresh. Eventually, most Nuvi mutants and later, the Drive models started using MTP or AFT to do this same thing. The handhelds at least through the Oregon 600 (which may well be my final geocaching GPS) were still essentially shutting down while connecting to USB for this same reason.

USB isn't  - and doesn't pretend to be - a network file system. USB mass storage actually exposes raw blocks on the device in SCSI command blocks (yes, really - and for an extra laugh, some versions of USB MSTO even exposed floppy drives as SCSI devices).

So software liie AFT acts like a tiny little TFTP server that reads and writes files (not blocks, though it may allow partial writes within files, such as for appending) where everyone talks to it and it alone is responsible for actually managing the storage device. (Well, it probably delegates that to lower levels of the OS, such as the kernel's own filesystem and journaling and below that, block level management) Unlike that "DADT" model where everything talks to a server (like AFT) though, in the era of removable media, we often want to be able to take the memory card to something ELSE and read and write it there, like mounting that memory card of pictures in your TV to share with others in the room. For THAT, we can't pretend that the code below AFT has just handled everything for us and we still need everything to be able to read and write the block level jibberish that the appliacations write to the filesystem. While we've created scores of successful filesystems, none are as ubiquitous as DOS's own FAT12, FAT16, and FAT32.  The relevant patents (complete with Ballmer-era shakedowns and litigations) have only very recently expired on those. So the filesystem bits (the etchings on those clay tablets) are useful to be able to read and write across devices. That's why we haven't all moved our external disks and our TiVos and whatever to EXT4, ZFS, and other, better designs.

So really, Garmin's engineers have two possible stances while building a device - they either detect the edge of an insertion/removal and USB attachment/detachment and they put the host device into a catatonic state where it can no longer access the card data (else things like the map it was displaying might be removed in the middle of a frame draw) or they add something like AFT to negotiate the access of BOTH the internal and external (host computer) access to the common media.

Software like GSAK has seen this train coming for years. So far, on dozens of models they've been able to turn off MTP mode and choose the "catatonic" model I've described. GSAK users have been doing this for a very long time.

Again, it's a lot of words, but hopefully this explains why these things are this way. (And, yes, I did formerly engineer this sort of stuff for a living..)

Edited May 7 by robertlipe

[+Mineral2]

I guess I don't quite see the problem since Garmin devices don't allow you to do anything while they are in mass storage mode, unlike other devices that still let you use them while you are tethered to a computer.
Well, here's hoping that iCaching builds in a MTP protocol to allow it to directly communicate and write data to the device/SD card.

[robertlipe]

Skimming topics, i decided to dip into the other remaining source of GPS expertise, the GSAK forum. (Duly noting that OP is on a Mac and this does them no/little good.)

https://gsak.net/board/index.php?showtopic=36383&view=findpost&p=279765

concludes that there is MTP voodoo within GSAK (!?!?)  Perhaps, this being a Microsoft creation as part of Windows Media, plinking around media files is easier there and GSAK somehow hooks to some kind of OS interface to do this.  (We've also had people declare the end of Garmin MTP about every six months since the original Nuvi 350, so I don't know if the 67 is really really the end or not...)

Still, when I see authoritative-sounding MTP Library things on the web saying things like:
 

Quote

"The official 1.0 specification for MTP was released by the USB Implementers Forum in may, 2008. Prior to this, only a proprietary Microsoft version was available, and quite a few devices out there still use some aspects of the Microsoft version, which deviates from the specified standard."

It seems quite likely that if Clyde came to me before May, 2008, I'd have told him to take off as it's MS-only and after May, 2008, I'd have told him to take off as sounds like the standard compliance is poor and reading things like https://github.com/libmtp/libmtp (search for 'kext') encouraging people to disable their USB mass storage devices, which requires unlocking several security checks, it just sounds like a bad idea even on MacOS.  (I can be a jerk like that...)

In that same doc, we have both:

"Windows Media Player apparently never close the session to an MTP device." ...

"The "Unix way" of running small programs that open the device, do something, then close the device, isn't really working with such devices and you cannot expect to have command line tools like the mtp examples work with them." 

You may be able to think of a program that very much embraces The Unix Way. It seems likely that you'd be deeply annoyed at GPSBabel running forever or requiring you to reboot between times of talking to your GPS. That's just incredibly dumb, but that's 2000's era Microsoft for you.

So even now, if they were to ask, the words "take off" may not appear literally in my response, but without someone else paying for the engineering to bring a quality MTP implementation into GPSBabel along with a device testing budget, I don't see it happening. Heck, Maybe Garmin had to develop one for Basecamp that's usable open source (hahahahaha).

Maybe iCaching is interested in developing some kind of code to talk to these things, but I don't particularly see this landing on my desk as an accepted action item without some divine intervention. (That happened on the 60 Cx...)

Since I hadn't finished reading the entire internet before posting that (workin' on it!) it possible that https://openmtp.ganeshrvel.com/ may be better, worse, or at least broken in a _different_ way.

It doesn't look particularly Mac-like for a Mac app,  but I have no vote.
Working + Homely > Attractive + Dysfunctional (This is also solid dating advice...)

Edited June 17 by robertlipe
Add OpenMTP