Celebrated several things these past few weeks — including Miranda's white coat ceremony!!

I've added syntax highlighting to the blog! Long overdue. Here's how I made it happen:

Setup

This site is a Next.js app. The blog posts are generated with the built in Static Site Generation feature. For each post, I grab all the urls to render and then they are constructed at build time:

import AlbumPage from '/components/albumPage';
import { getAllPosts, getAlbumBySlug, getAlbums } from '../lib/api';
import { getPostBySlug } from '../lib/markdownAccess';
import PostPage from '/components/PostPage';
import markdownToHtml from '../lib/markdownToHtml';

// The Main Component
export default function SlugPage({ post, album }) {
  if (post) return <PostPage post={post} />;
  if (album) return <AlbumPage album={album} />;
}

// Get static props - gather required page data based on page
export async function getStaticProps({ params }) {
  
  // . . . 
  
  const post = getPostBySlug(params.slug, [...);

  if (post) {
    return {
      props: {
        post,
      },
    };
  }

  return {
    notFound: true,
  };
}

// Get the static paths for all posts and pages
export async function getStaticPaths() {
  const posts = getAllPosts(['slug']);
  const albums = getAlbums();
  const slugs = [...albums, ...posts].map((contentObj) => contentObj.slug);

  return {
    paths: slugs.map((slug) => {
      return {
        params: {
          slug,
        },
      };
    }),
    fallback: 'blocking',
  };
}

The post object contains the raw markdown and meta data for the page. All of the site's pages are built from that markdown and are rendered to JSX through this component:

import markdownStyles from './markdown-styles.module.css';
import Markdown from 'markdown-to-jsx';
import Link from 'next/link';
import Image from 'next/image';
import NextLink from './NextLink';

export default function PostBody({ content }) {
  return (
    <div className="markdown">
      <Markdown
        options={{
          overrides: {
            a: NextLink,
            img: BlogImage,
          },
        }}
      >
        {content}
      </Markdown>
    </div>
  );
}

// const BlogImage = (props) => <Image {...props} width={800} layout="fill" />;
const BlogImage = (props) => (
  <a href={props.src} target="_blank" rel="noopener noreferrer">
    <img {...props} />
  </a>
);

Markdown to JSX is doing the heavy lifting of rendering my markdown annotations to html. I've also plugged in a few custom overrides to make use of Next features, such as the NextLink to handle routing through the app, as well as an img override to open in a new tab by default.

Adding In Highlight.js

Highlight.js is a flexible library that can do exactly what I'm looking for, both on the client and server.

Since I'm building static pages, I'll reach for their server implementation to call:

html = hljs.highlightAuto('<h1>Hello World!</h1>').value

I could use their client side approach, wrapped up in a useEffect. However, that adds to the js bundle sent down the wire. Not to mention, I'd get this ugly flicker effect once the styles kicked in.

So, I'm opting to build another override.

Markdown renders code in a <pre> and nested <code> tag. So I'll add my own components to plugin the synchronous syntax highlighting:

First, importing highlight.js and adding my override:

import hljs from 'highlight.js';

export default function PostBody({ content }) {

  return (
    <div className="markdown">
      <Markdown
        options={{
          overrides: {
            a: NextLink,
            img: BlogImage,
            pre: Pre,
          },
        }}
      >
        {content}
      </Markdown>
    </div>
  );
}

And then writing my custom components:

const CodeBlock = ({className, children}) => {
  children = hljs.highlightAuto(children, ['java', 'javascript', 'python', 'react', 'yaml']).value;
  return (
    <pre>
      <code dangerouslySetInnerHTML={{__html: children}} />
    </pre>
  );
}

const Pre = ({children, ...rest}) => {  
  return <pre {...rest}>{children}</pre>;
}

Violà! The colors you see above are thanks to these changes!

Lucy's been listening to my teenage emo CDs! Now she's all moody.

Purchase on 🤘 Bandcamp and Listen on 🙉 Spotify or any of your favorite streaming services!

Listen on Youtube

Been enjoying sightreading short and sweet 5-finger pieces like this.

A study of my destop companion 🦖

Of all the pages of official docs in Python, the page on sorting by Andrew Dalke and Raymond Hettinger may be my favorite. Clear, gradually continues on in complexity, and provides ample examples along the way.

Here's my situation this week: Simple enough, I needed to sort a list of dictionaries by a property on those dictionaries:

data = [
    {
        'name': 'Chris',
        'count': 5,
    },
    {
        'name': 'Lucy',
        'count': 3,
    },
    {
        'name': 'Clyde',
        'count': 3,
    },
    {
        'name': 'Miranda',
        'count': 10,
    },
]

To sort by the count, I could pass a lambda to access the property on the dict:

sorted(data, key=lambda x: x['count'])

Say that the counts are equal as it is with Lucy and Clyde in my example. If so, I would want to sort by the name.

Returning a tuple covers that:

sorted(data, key=lambda x: (x['count'], x['name']))

To reverse the order, there's a named property for that:

sorted(data, key=lambda x: (x['count'], x['name']), reverse=True)

Problem all sorted out!

match = next(x for x in db_data if x["PropertyId"] == parsed_incoming_data["PropertyId"])

Breaking it down:

• next() returns the first value of an iterator. In subsequent calls, it would return the following item.
• next requires an iterator. An iterator yields number of objects on the fly. This is different from a list which contains and stores values. Lists and tuples can be converted to iterators with the iter() method.
• In my example above, the list comprehension x for x in db_data above yields an iterator, covering our type requirement for next.
• We're filtering by matching another value: if x['PropertyId'] == parsed_incoming_data['propertyId]

Voilà! Filtering for a match in one line.

Listen on Youtube

Enjoying the space of these chords 😌

An outdoor study

As Jamison Dance put it in this week's episode of the Soft Skills Engineering podcast: "It takes just one more level of abstraction to be a great engineer!"

A task at work this week had me looking to decouple one of our components that uses a third party library. Let's say it's a bar graph that uses something like Chartist.js. We want to be able to reuse this chart in both full page settings as well as where it's a widget inserted into a page. In one case, clicking a bar would open a tooltip. In another, it may link to another page.

Normally, here are my considerations for doing that:

1. From the base component, elevate as much of the logic as possible. The child should essentially be a "view" component only concerned with rendering data.
2. Pass any interactivity down through callbacks such as "onClick", "onMouseEnter", etc.

That works fine in this example below:

const BarGraphContainer = (props) => {
    const dispatch = useDispatch();

    const onClickBar = (segment) => {
        dispatch(actions.openModal(segment));
    };

    return (
        <BarGraph
            onClickBar={onClickBar}
        />
    );
};

The Problem

In this instance, the base component is handling a third party library for initialization, setup, and much of the internal interactions. In some cases, I want to control firing off an event in that internal library (ex: opening a tool tip on click). But in other cases, I want an external behavior (linking away or opening a modal.)

Passing Context Up Stream

An interesting solution I came up with was one that I had seen in other libraries and ecosystems: Passing context to the callback.

When considering passing callbacks in react, a simple use case typically only passes an event object.

const onClick (e) => e.preventDefault();

However, if I need access to the internal API of our third party library, I can pass that up through the callback as well. Even batter, I can abstract most of the internal workings of the library with a wrapper function. Take a look at the example:

const BarWidgetContainer = (props) => {
    const onClickBar = (segment, graphContext) => {
        graphContext.renderToolTip(segment);
    };

    return (
        <BarGraph
            onClickBar={onClickBar}
        />
    );
};

Here, the renderToolTip function likely has a great deal of logic specific to the library I'm interfacing with. At a higher level, though, I don't have to worry about that at all. I can simply call for that functionality if needed from a level of abstraction.

Use Cases

As mentioned, the abstraction is great for providing flexibility without complexity. Consumers of the component can interface with the internals without getting into the weeds of it.

A major con, though, is the added decoupling. In most cases, this could be seen as an anti-pattern in React given the uni-directional flow that's preferred in the ecosystem.

Considering these points, we ultimate decided on another solution that allowed for the parent to child data flow. It makes the most sense for our situation since it keeps our code cleaner. Realistically, we're also only using this component in a hand full of use cases.

But why did I write this up, then? I'm keeping the pattern in my back pocket. Situations where I can see this being useful are in broader use cases. Say that instead of our internal React component, this were part of a larger library consumed by more users. The trade off of coupling for abstraction and flexibility might make sense in a more widely used library. That's likely why it's a frequent pattern in open source tools, after all.

It was a fun experiment this week! Saving this pattern for another time.

Listen on Youtube

Trying out stride piano! All without looking at my hands 🙌

Had my breath taken away by the background painting work of Merle Cox, Ray Lockrem, Robert Storms, and W. Richard Anthony in Disney's Fantasia's Night on Bald Mountain. Decided to explore the colors in a different context!

Sharing a bit of initial research for a serverless project I'm working on.

I've been wrestling this week with a challenge to coordinate a cron job on AWS Lambda. The single script running on Lambda is ideal. There's no server overhead, the service is isolated and easy to debug, and it's cost effective. The challenge, though, is how to scale the process when the run time increases.

Lambda's runtime limit at this moment is 15 minutes. Fairly generous, but it is still a limiter. My process will involve web scraping, which if done sequentially, could easily eat those 15 minutes if I have several processes to run.

The process looks something like this so far:

1. Initial function start, get a list of pages to crawl.
2. Sequentially crawl those pages. (separate lambda function called sequentially)
3. After page crawl, send an update on the status of the crawl and update the DB with results.

Simple when it's 3 or 5 pages to crawl for a minute each. But an issue when that scales up. Not to mention the inefficiency of waiting for all processes to end before sending results from the crawl.

This would be a great opportunity to lean on the power of the microservice structure by switching to concurrent tasks. The crawls are already able to be sequentially called, the change would be figuring out how to send the notification after the crawl completes.

To do this, each of those steps above can be broken up into their own separate lambda functions.

Once they're divided into their own serverless functions, the challenge is too coordinate them.

Self Orchestration

One option here would be to adjust my functions to pass state between functions. Above, the 1st lambda would grab the list of pages, and fire off all instances of the 2nd lambda for each page. The crawler could receive an option to notify and update the db.

It's a fine use case for it! With only three steps, it wouldn't be overly complicated.

To call a lambda function asynchronously, it simply needs to be marked as an "Event" type.

import boto3

lambda_client = boto3.client('lambda', region_name='us-east-2')

page = { ... }

lambda_client.invoke(
  FunctionName=f'api-crawler',
  InvocationType="Event",
  Payload=json.dumps(page=page)
)

Step Functions

Say it were more complicated, though! 3 more steps, or needing to handle fail conditions!

Another option I explored was using Step Functions and a state machine approach. AWS allows for the ability to orchestrate multiple lambda functions in a variety of patterns such as Chaining, branching, and parallelism. "Dynamic parallelism" is a patern that would suit my case here. Though, I may not necessarily need the primary benefit of shared and persistent state.

The Plan

For this use case, I'm leaning towards the self orchestration. The level of state being passed is not overly complex: A list of pages from step 1 to 2, and then a result of success or failure from step 2 to 3. The process has resources in place to log errors at each step, and there's no need to correct at any part of the step.

Next step is the implementation. To be continued!

Listen on Youtube

Walking that bass! Trying out playing without looking at my hands as well.

Followed this great video tutorial for the tune

Really enjoyed Netflix's Avatar! Such a feast for the eyes 🤩