|
There are 17 blog entries within the category of Software
A few days after releasing the new blog format, I found there was a small bug with the grouping of older month/year combinations. I've since fixed that code and everything seems to be working as expected now.
I should also mention that there has been a great deal of work that's gone into maintaining the page Sitemap. This isn't intended to be used by any regular person, but rather should help the various bots to better index and catalogue the full extent of the site.
|
I've been meaning to write a post for a couple of weeks now, but I've been especially busy with work!
I've been fortunate enough to be implementing all sorts of new innovations at work and so I've had to balance my development time accordingly. But between the massive code changes to this site (and ultimately the software that runs it), the various publications that I've been working on, the handful of TikTok channels I've started managing, the music work I've been trying to stay on top of, and the numerous travel plans I've been working on (namely to Alaska, Hawaii, Maine, and Africa), things have been extremely busy!
While the site might not look super different just yet, all of the new development code has been merged into the main code branch and I can absolutely say that the changes are plentiful.
One of the biggest changes to come out of the new release has been the addition of my own Blog management software.
How I Originally Modified Wordpress
When I first wrote this software back in 2005 (at which time the project was called staticMOD), I actually had taken the time to write my own blog management software. Blogs hadn't been around all that long and they wouldn't last much longer with the advent of YouTube mostly taking over the space. So although staticMOD relied upon a custom blog interface, the successor codebase. called openFace, did not. Instead I simply modified the backend of Wordpress and used part of it inside of my administrative toolset.
None of my site was actually running on Wordpress. Rather, there was just a nice text editor built into it and so I customized it so that the entries would be saved into my own personal database with my own personal flair. It's worked extremely well for the past 15 years, but I've been wanting to officially move away from it and so I finally have. My goal has been to have as few 3rd party applications as possible.
The Process of Custom Blog Management
I started making these changes in the summer of 2021, but then had to put the project on hold for a little while. I picked it back up in November and made huge strides - generally getting to work on it most every evening. But with many more personal downturns still ahead, I ultimately had to put the project on hold for a few months - which I did. When I finally came back to all of the coding around March of 2022, it's pretty much been a steady sprint to the finish line. And here we are.
There are definitely still some things that I'm working out, but all in all the site is much easier for me to manage. I now have the ability to easily start integrating all of my social media into my own site. I don't think we've yet reached a time whereby people truly value that, but some day they will.
Additional Tech Highlights
There are two additional tech highlights that I'll mention as they've both been incorporated into my software.
In December of 2021, I put a ton of work into a language processing tool. It was essentially designed to be fed any kind of communication between two people and to analyze every little nuance of said communication. I thought it was a pretty cool tool. While I had fun building the project, it ultimately proved to be a huge waste of my time and effort (the project did take me a ridiculous amount of time to make and like most of my projects was coded entirely from scratch).
But as I'm always one to find the silver lining, I took that software and instead retooled it for openFace. I haven't released it just yet, but it has the ability to analyze decades of information that I have and break that information down into much more meaningful pieces, all of which will have accompanying timelines.
And then in April/May of 2022, I finally was able to build my own facial recognition tool. I've only recently found a good script that will allow me to use this, but my hope is that I'll be able to incorporate this into openFace and ultimately have my entire photo (and later video) library automatically catalogued for me with primitive AI.
More Changes Coming
There are still a host of changes that I haven't rolled out yet. As I had previously written, all of the photos and videos are now in a distributed cloud network. They load very fast and much more importantly, I finally have an infinitely scalable filesystem. I will definitely be releasing these assets more and more in the coming weeks and months.
|
After many, many months of code changes, I'm delighted to announce that this site is about to get a pretty massive overhaul. The changes won't necessarily be visible to the end user (eg: it will still look the same), but the infrastructure that powers the site will be significantly different. It will be much more powerful, much faster, and much cheaper for me to operate and maintain. The end result is that there will be a lot more content (which I don't feel like I'm really lacking in the first place).
What's the timeline?
I started working on these software changes in early November of 2021. It's a passion-project so I only work on it when I have some spare time. I wound up taking some time off from the migration in January and early February, but then got back to full speed in early March. Since then, I've managed to get everything more or less where it needs to be. It'll still be a few more weeks before I can actually "flip the switch", but as I've been wanting to make this change for a couple of years now, I couldn't be more thrilled to be getting so close.
What changes are coming?
The short answer is that it's a massive infrastructure overhaul. Without going into too many technical details, all of the backend software, operating system, database, and processing infrastructure has been upgraded. This should result in the site running much faster and should allow me to add several pieces that I've had in the pipe for years (detailed photo statistics, geolocations, and videos, just to name a few). It's also going to result in all of my media being distributed on networks around the world. While this will likely have the result of my site being indexed much more heavily and routinely, it should also reduce media loading times rather significantly. I've been testing them for awhile and everything seems to be on par with what I'd expect.
Which brings me to the change I'm most excited about: I'll finally be able to put my entire video library online! When I first built the original software back in 2006, I thankfully had the foresight to create full video support. That might not seem like a big deal today, but archiving digital videos back in 2006 was an extremely daunting and difficult technical challenge. I continued supporting my video management with several major upgrades over the years, but it eventually got far too difficult to manage. The bottom line was simply that we were creating digital videos at a rate that was just too cumbersome to keep up with. Not to mention, storing videos is expensive. I stopped trying to make all of my videos available some time around 2014 or 2015. That's a pretty big gap now.
Once the migration is completed, I'll finally be able to start putting these online and properly managing them. I've been using YouTube's API for years to automatically place videos on their site, but I'm hoping that I'll be able to do the same with TikTok.
|
As a small technical note for my blog, I've been busy writing a ton of code to fix some of the issues that have recently come up. I'd been working on this over the past several months, but took a bit of a break in January while tending to other things. I've written about this rather extensively before, but most of the problems stem from Apple stupidly reorganizing the way they store photos within the iPhoto app.
While they used to keep a tight directory structure on your photos disk (making it extremely easy to RSync to one's personal server), they are now using some kind of hex-based directory structure and simply add all of the photos into these folders using $0 through $F (eg: ~/0/, ~/1/, ... ~/F/). Worse still is that they include the photos in the directories seemingly at random.
This makes it extremely difficult to find your new files and move them where they need to go. Yes, you can use the export feature, but this is SUCH a slow process and makes absolutely no sense for storing your own photos on your own system. It's absolutely bonkers that they've adopted this new structure.
Of course this is all in line with Apple taking the blue ribbon for "most evil tech company". They're quite clearly doing this to make it more and more difficult for people to use software outside of the Apple realm thus ensuring that people stick with iPhoto (and much more insidiously, their Cloud services). I've been writing about this for nearly 20 years now and this is just the latest example of it.
This required me writing code that would look at the MD5 checksum for every single file and then compare those to the files that I already had on my server. The ones that were missing were copied into one of the new directories and processed.
I still have a ton of photos needing to be sorted, but it's coming along in my spare time.
|
Anyone that has the new MacBook with the TouchBar knows that it is simply THE worst piece of hardware that Apple has ever released. Anytime I get a new MacBook there is always that period of time where you have to adjust to the new keyboard. The first time I ever experienced this I cursed them for changing it, but I've learned over the years that the changes are always for the better and after a week or so, my brain is entirely wrapped around the new layout and it increases productivity.
That is simply NOT the case with the new MacBook. I've waited and waited for that eureka! moment to arrive and it simply does not. After reading countless blog posts on the issue, it seems that I am far from the only person to experience the problems that I have.
One of the MANY problems that I experienced was with respect to trying to launch my spotlight. For as long as I can remember, this was a simple combination of Command+Spacebar. I specifically keep my dock as empty as possible knowing that I can pull up any program on the fly through the Spotlight. Unfortunately, the new Apple seems to stutter for lack of a better word.
More times than not I press Command+Spacebar and nothing happens. Usually pressing it a second time will do the trick, but sometimes it takes even a third. Why? Why would this possibly be the case?
Well after some research, it turns out that this has to do with Siri's functionality waiting to see if she is being called upon or not. Note to Apple: she isn't - ever - being called. I just want the Spotlight to open.
So here's the fix:- Open your System Preferences
- Select Siri
- The first option is "Keyboard Shortcut"
- Change the dropdown so that it reads "Press Fn (Function) Space"
Once you do this and save the system preference, you'll find that the problem immediately goes away. I am hoping to compile a list of such changes that are necessary to get the computer somewhat back to what it used to be, but unfortunately I don't think the new MacBook is ever going to work as well as its predecessors, regardless of how many tweaks I make to it.
|
 The YouTube Logo
I'm not referring to police records and tax documents. I'm referring to that picture of you where you've had six too many to drink, your eyes are bloodshot, and there's still some puke running down your lip from when you last vomited. These are moments that we try dearly to make disappear, but they're not going to disappear. The second an iPhone or Android snaps a picture like that, not only is the content saved in an archive forever (yes, even if you delete it), but the META-data attached to the photo is frightening. There is a full story of exactly where you were, what you were doing, how long you were there for, and everything else in between included in that META-data.
I wouldn't exactly say that my attitude followed the old cliché of, "If you can't beat em, join em", but it was along those lines. The idea of my website was that if this is where we're headed, then perhaps it's just best to embrace those moments and own them as your own. So what if I had too many to drink one night. It happens. If anybody is going to own a picture of that moment, I'm going to. I'm going to be the one to make it publicly accessible for all to see. If someone should ever try to use it against me, I'm going to politely remind them that they got the image from my own personal archive. There's a power to this and I encourage everyone to embrace it.
That's the back story.
Along with that has come hundreds of hours of video footage that I've recorded over my lifetime. FINALLY, I have finished the segment of my site whereby I can archive all of my video footage not only on my own personal website, but on YouTube.
Google, like with most of their tools, has created a wonderful API for connecting with your YouTube channel and uploading videos to it. I've taken full advantage of this and in the past month or so have uploaded about 1,500 videos to their service as part of the initial experiment. Now that I finally have the tools to do it, I probably have 200-300 more hours of video that I will fully digitize and send to YouTube.
My process is simple. I start by placing all of my media content in the regular directory it falls on within my server. Think of it like a giant iPhoto gallery. It's just an enormous directory tree with about 90,000 unique memories in it (as of this writing). From there my servers run a series of automated processes that I've developed over the years. Photos are converted into different sizes and resolutions as needed by my software package. Videos are converted into OGG format (open-source video for the web) so that I can display them using HTML5 on this site. Thumbnails from the videos are automatically extracted and organized. Each entity that is created within my system is given a unique ID and catalogued within my database. There is a lookup table that will cross reference any of these unique IDs in the entire system with a specific file type (photo, video, etc.)
Focusing solely on the videos, at this point in my process there exist videos on my server that have thumbnails, that have been converted to a useful (and view-able) web format, but that lack any titles or description. I quickly scroll over the video, provide it with a title and a description, and so long as I'm okay archiving it within YouTube, I check a box and it gets marked for upload.
A standard CRON job on my server is routinely scanning the database for videos that have been approved, but not yet uploaded. Once it finds such a video, the YouTube API is invoked and the video gets uploaded to YouTube. All of the information about the video that has been populated in my database (such as the title and description) is sent over with the video for YouTube to archive. The date/time that the video goes over is logged in my database as is the unique video code that YouTube provides. From there I can reference any video in my personal catalogue by looking up the YouTube ID myself.
While I've not yet configured the videos as a YouTube plugin on the site, this is the obvious next step. Not only does it take cost off of my server, but YouTube's servers are likely significantly faster than mine.
That's the gist of how it works. I've wanted to do this for about 7 years now. I'm delighted to have finally finished the coding that makes it work.
Incidentally, if you go to my YouTube channel, you find thousands of videos, all of which originated on this site.
|
|
