Org:jrbeverly on jrbeverly

swagger-golang-bazelgen-exp

Thu, 19 May 2022 00:58:15 +0000

Golang & OpenAPI Spec

Fiddling with an idea of generating models from the OpenAPI spec for YAML.

Notes

Bazel model works, but would want to formalize as a proper build system
Model of pkg/apis/... is a good direction, version selecting will need to be a thing though
What about YAML Support?
Is swagger the expected way forward for this kind of thing?
Should the generation of files be done as a bazel run or a bazel build?
Generation should really commit against the repository, that way we can be more confident about the code results
Works much better to have all components available

graphql-golang-note-check

Wed, 18 May 2022 23:21:25 +0000

Fiddling with gqlgen

Experimenting a bit with gqlgen for generating GraphQL code from spec.

Notes

Would prefer to move gqlgen.yml and schema.graphql into a spec/ directory (or other areas)
Installation method with tools.go - not sure about this, static binary is preferrable for my usages
Generated models are pretty solid, similar to swagger
Resolvers is nice, but what about partial updates?

I’m not entirely sold on this pattern. Feel like I’d prefer more flexibility with how the models and various components are defined. Almost interested in the idea of instead of using annotations like json or yaml, leveraging generated code for rendering the JSON/YAML components. Then isolating the various elements of what the system is expected to do, things like:

react-xstate-machines

Wed, 18 May 2022 23:14:27 +0000

State Machine for Confirmation Dialog

Running through the workshop example of Build A Confirmation Modal in React with State Machines

Notes

In principal, like the idea of representation this kinds of logic “Flows”
Usage of strings for state is less than ideal, almost would want it to be objects
Pattern of constructing a “flow” then making use of the “flow”
Potential opportunities with systems like codegen
Possible ideas for test evaluation with the states
Not sure about this library, needs opportunities for isolation
Feels like it doesn’t fit with the View -> ViewModel -> Model concept I was thinking of for State Machines
The ‘dispatcher’ works similar to what a wrapper over “Modal” would have, what benefits does it bring?
Consider looking into eventing systems instead, they seem to better encapsulate this idea without strictness of the “machine structure”
Maybe this could work better when combined with message passing + code generation?

exp-pulumi-lambda

Wed, 18 May 2022 00:40:54 +0000

Experimenting with Pulumi

Experimenting with the pulumi examples from https://github.com/pulumi/examples , and the options to have

Notes

Enabling Bazel wtih this model encountered some pain points
Pulumi is capable of performing operations like Docker Build / Referencing lambda binaries
The change source pattern (git.dirty, git.author, etc) for volatile status is a nice pattern
Building of artifacts should not be the responsibility of the deployment model
Pulumi takes over the context element of deployments, and use that for deployments

Pulumi has some nice elements too it with respect to be able to use code to create the deployment, as well as a good web/console interface. However I’m not sold on the idea of these kind of “dynamic” systems, and instead prefer the style of an application model. This way a series of concerns can be baked into “Configuration” that is then acted upon, similar to Terraform.

golang-gin-gitpod

Wed, 18 May 2022 00:23:29 +0000

Golang Gin & Gitpod

Fiddling with the Dev experience of Golang gin within Gitpod

Based on gitpod-io/go-gin-app

Notes

Prefer the pattern of combining this with cobra
Server rendered HTML is pretty straightforward
Returning simple JSON pretty straightforward
Can be combined with GRPC + Proto
What about swagger for generating the modules?
Considerations about how to organize the elements (cli, server, routes, models, database, etc)
What about logging for this?
Gitpod has occassionally issues with the import of gin
References for things like templates/ at the top level is nice
Log while running is excellent

Overall positive.

reminders-miragejs

Wed, 18 May 2022 00:07:08 +0000

MirageJS Tutorial

Running through the (MirageJS Tutorial)[https://github.com/miragejs/tutorial] case

Notes

If generated from a schema, the system is essentially
- Backend (proper)
- Backend Mock (in-memory)
- Backend JS Client (proper)
- Backend JS Client Mock (in-memory - integrated with MirageJS)
How could this be integrated with code generation from a specification?
Does this assist with local development?
Test scenarios in an pseudo-E2E case?

manim-exp-video-generation

Tue, 17 May 2022 02:10:19 +0000

MAnim Experimentation with Video Generation

Experimenting with the MAnim library for generating video animations.

Notes

Solid library that works really well at what it does
Scenes feel very “on-track” in that it is a sequence of steps one followed by the next
Python allows for re-use to create common elements like an isometric file
Works great for scenes where the emphasis is on the core “shapes” and animation

Don’t think this fits with the use-case I’m after. Would make more sense to explore as a unity “game” that is really just an interactive cutscene. As that would allow for elements of the scene to still remain dynamic (animated) while the core “state” of the animation remains at a fixed point in time.

react-wasm-babel

Tue, 17 May 2022 02:05:18 +0000

React Webpack with Rust WebAssembly

Fiddling around with an opinionated example for Webpack builds with WebAssembly.

Source code for fractal is based on https://dev.to/brightdevs/using-webassembly-with-react-1led , and the repository templated by https://github.com/Fallenstedt/wasm-react-webpack-template .

Notes

WASM Build works pretty well with rust
Rust is a solid option for getting webassembly integrated
Golang was considered, but previous experiments weren’t as desired
Makefile as an entrypoint is preferrable than using yarn ...
Embedding the generated packages within the front (e.g. under pkg/) allows for local refs
When does the build perform certain targets? E.g. resolution of paths vs copying of static files?
Searching for options of individual target execution didn’t yield much (using wrong search terms?)
Webpacks plugin model isn’t really the desired execution model, prefer graph-driven like buck/bazel
Webpack plugin dumping the files just by directory path isn’t ideal, would prefer to ref (e.g. //pkg/fractal)
Loading WebAssembly requires a bit of pre-amble, could probably be made into a <project>-commons for UIs responsible for loading it
Usages of them are pretty nice, opportunities for generated API/Schemas/State Machines/State Models

I’m not sure on this one. I get the benefits that come with the webpack ecosystem, but had multiple issues with the way its goes about it in comparison to patterns like Bazel/Buck. I like this pattern more for combining various languages to create the final result, such as having a rust library for certain functions or components. Splitting it away from the frontend makes it so they can be designed in more of a “unit-test” way. Possible options might be to encode a state machine within one of these libraries, then have the web interface act on this, like a View => ViewModel => {Model} where the ViewModel performs queries into the state machine {Model}.

vuln-disclosure-policy

Fri, 13 May 2022 03:25:29 +0000

Vulnerability Disclosure Policy from Dioterms

Exploring leveraging dioterms and policymaker for creating vulnerability disclore policies for a website.

Notes

DNS is related for the deployment of the website (_security)
Entry within the /.well-known/ root of the domain (example.com/.well-known/security.txt)
Security entry for the domain (example.com/security)
If the application is located within example.com/app/... (e.g. index.html), then the top level domain elements can be “procedural”
Construct the webpage into a bundle (website.wbn), publish it to the “deployer”, which can then handle the top level elements
References can still exist within the app (/security, /.well-known/...), known to the website manifest
Website manifest allow it to enforce expectations about required top-level components
Distributable/Sharable webpages can combine/merge these components (e.g. website.wbn, website.manifest, website.policy) with organization (or overwrite)

Acknowledgements:

bazel-terraform-conftest-experiments

Wed, 11 May 2022 02:21:49 +0000

Experimenting with ConfTest, Terraform & Bazel

Experimenting with using Bazel to handle the build & execution of Terraform files, while providing means of writing tests against the terraform with conftest. Fiddling with the idea of having local tests against the configuration, as well as tests against the terraform plan.

asdf

Tue, 10 May 2022 22:33:19 +0000

asdf

Manage multiple runtime versions with a single CLI tool, extendable via plugins - docs at asdf-vm.com

asdf is a CLI tool that can manage multiple language runtime versions on a per-project basis. It is like gvm, nvm, rbenv & pyenv (and more) all in one! Simply install your language’s plugin!

julia-with-jupyter-notebook

Thu, 07 Apr 2022 03:14:33 +0000

Julia Jupyter Notebook

Experiments with working with Julia & Jupyter Notebooks.

Notes

Installation of Jupyter, Conda + Julia (+ packages) onto a gitpod image
Makefile for common actions working with the notebooks
Validating some basic cases working with Julia for generating plots & other bits for the visualization

gitpod-jupyter-notebook

Thu, 17 Mar 2022 23:41:20 +0000

GitPod Jupyter Notebook

Validating working with Jupyter notebooks in a GitPod environments

Notes

Support for preview or open in browser mode
Initial provisioning of the image takes a bit, pre-baked likely would help in this area
Jupyter lab is another option
Makefile is a good option for acting as an entrypoint for common commands
Outstanding questions still exist for multi-notebook repositories like the learning repo

pushgateway-compose-setup

Tue, 15 Mar 2022 22:33:21 +0000

Pushgateway Compose Setup

Simple code setup for spinning up Pushgateway, Prometheus & Grafana for validating lifecycle pushgateway metrrics.

Notes

Metrics published to pushgateway are collected by Prometheus
Prometheus is enabled in Grafana for queries
Grafana datasources & dashboards are configured from the provisioning directory
The script publish.sh can be used to publish the metrics into the system
Dashboards in grafana/dashboards can be configured with other tools for construction
Dashboards could be standardized, then shared into other sources
Tools can be configured for publishing in these scenarios

aws-exp-organizations-policy

Sun, 06 Mar 2022 21:32:19 +0000

AWS Organization Structure Experiments - Mirrored Organizations

Experiments with AWS Organization structure and potential SCP policies.

Notes

The entire organization unit hierarchy shouldn’t be a single entity for mirroring. Makes it difficult to evaluate in “isolation”
Entire organization mirrors can work with the SCPs, but internal permissions (e.g. S3 Bucket) still might have issues
Organizations should include a uniqueness component to allow for constructing a new version (& prototyping)
SCPs seem like they would benefit in cases where there is a sort of “State machine” in the SCP
State machine examples are “During provisioning of account, need to create IAM Users, but from then on no users should be created”
Account boundaries for services as a way of strictly locking things makes it easier to have DenyKMS and other such policies
Region denies only apply after provisioning, as we need to purge the “default VPCs” created when an AWS Account is created (+ any other “default” resources)
AWS Password Policy / AWS IAM Account Name / Etc are all good examples of something that should only need to be provisioned “once”
SCPs give a potential idea for the concept of “Immutable AWS Account Infrastructure”, that require you to create a new AWS Account (+ migrate resources) rather than edit them
Sandbox/Staging organizations can contain the developer workloads that are for sandbox/development
Developer workloads should be contained within accoounts that can be created/decommissioned on a release schedule (see ubuntu - Bionic Beaver, Focal Fossa, Xenial Xerus)

More investigation is needed into this idea, as the exact “concern” that this kind of structure & SCP policy layout will handle is kind of vague. Although grouping AWS Accounts and associating tags with them can be useful for things like data residency / storage compliance, its not immediately clear how this design maps to the “problem” itself.

github-pullthrough-mirror

Fri, 04 Mar 2022 04:32:17 +0000

Release Mirror for GitHub Releases

Lightweight experiment for mirroring GitHub releases into a file store system like Minio or AWS S3.

Getting Started

packages/

Contains the tool definition files for each of the mirror repositories (tool.ini), and the computed checksums for the downloaded files. In an actual use case it would be better to store the checksums externally from this repository, allowing this one to act purely as an executor, and the other repository acting as a “record” of the known-good commit SHAs (as well as .sign/.ack files to chain “trust”)

github-config-in-code

Wed, 02 Mar 2022 02:14:31 +0000

GitHub Configuration in Code

Fiddling with the configuration options available for GitHub, while encoding the properties in the .github directory.

Getting Started

This repository

The concept is to create an almost “self-contained” repository, that includes within hidden directories like .github/.aws/.azure/.gcp/etc that represent almost “interfaces” between the repository and external services that would act on it. This way rather than having the repository rely on assumptions about how it is configured, it is instead providing all the baseline elements for any supporting infrastructure (e.g. operators) to provide these integrations themselves.

github-app-for-code-change

Wed, 23 Feb 2022 00:11:59 +0000

GitHub App for Generated Commits

Running through a bunch of things to be done with this

Notes

Is it possible to push empty commits, which would need to be handled
Commits should be crafted first, then attempts being made to apply the change
Would need to have better visibility into the “crafted” files that will be committed
Does this really require a GitHub Application? Feels like this could be decoupled
Need to ensure that requests with the API go through the proper retry processes
Need to ensure that the configuration is split from auth, so its easier to rotate the auth without fiddling with the config
Should the app really be responsible for the work-component of crafting the change?
Should all changes be made as a pull request + request to merge from the GitHub App? This would avoid conflicts as much, and let GitHub be responsible for changes?

Does this kind of thing really need a GitHub Application? Feels like this is something that can be split away from GitHub as a kind of entity that is responsible for the following actions:

gpg-artifact-sign-exp

Thu, 27 Jan 2022 02:07:13 +0000

GPG Artifact Sign Experiment

Minor experiment with a shell script for signing artifacts that would be generated from a build process.

Notes

Build tooling can support multiple checksum algorithms (sha256/sha1/md5/sha512/etc)
Docker Container Trust (DCT) didn’t fit with the usecase/portability desired
GPG is the standard way for doing this (can this be packaged into something more portable?)
Design should aim to be agnostic of GitHub Releases or any other platform
Build tooling likely doesn’t need to understand the concept of “signing” (or should it?)
If build tools understand signing, does that mean there need to be ‘Developer Keys’?
- Is this really just as just a matter of familiarity?
The CI/Build process is responsible for signing files to assert that “it” was responsible for the build (e.g. not just dev artifacts published to S3)

bevy-rustlang-example-window

Thu, 27 Jan 2022 02:02:45 +0000

Rusy Bevy Baseline

Fiddling with one of the Bevy examples for provisioning a window with Bevy.

Notes

Initialize setup required installation of libraries like: libasound2-dev libwebkit2gtk-4.0 libudev-dev mingw-w64
Cross-compilation works, but associated GitHub Issues seemed conflicting
Earlier version of bevy had issue with missing audio driver (devcontainer) failing the build

Don’t think will continue with this for now, maybe investigate later.

exp-webassembly-golang-bazel

Thu, 27 Jan 2022 00:52:56 +0000

Experiment - WebAssembly Golang + Bazel

Experimenting with some issues encountering with WebAssembly, Golang & Bazel

Notes

Confirmed issue with syscall/js in the basecase with using goos and goarch (toolchains passed as orgs better option?)
Using a genrule sufficient to workaround the case
Base case with a simple calculator, using just base HTML
js.Value conversions, framework wrapper to exist to handle the interop?
Directory layout of cmd/ and app feels a bit off, but does help keep the bits separate

packer-overwrite-motd

Thu, 27 Jan 2022 00:27:16 +0000

Packer Overwrite MOTD

Overwriting the MOTD of pre-baked AMIs using Packer

Notes

Message files are located in /etc/update-motd.d/
Existing ones can be purged and replaced with a fixed one
Likely want to keep the status components (source from existing, overwrite)
Scripts need to be executable, and include shebang
Current packer requires AWS for EC2 create/snapshot, can we avoid?

This works well for setting up a baseline for the AMIs. Is there a way we can perform this Packer construction without needing to be connected to AWS? As at the moment this more resembles “Assembly” than “Build”. Having a daemon (virtualbox/hyper-v) is less optimal, but if it can be solely run on any machine with the installed tools, then that would be preferable.

cobra-cmd-with-docs

Wed, 26 Jan 2022 04:11:54 +0000

CobraCMD with GenMarkdownTree

Experiment with the GenMarkdownTree method available with cobrago.

Actions

The command line utility can be executed by running:

bazel run //cmd/cobradocs version

The markdown tree for the docs can be generated by running:

xterm-for-cmd-as-site

Wed, 26 Jan 2022 04:07:04 +0000

XTerm for Terminal as Browser

Experimenting with the idea of a minimum environment for running terminal applications in browser. In essence, allowing a user to navigate to example.com/terminal to view a terminal version of the sites API. With the appropriate token & other bits provided from the browser session tokens.

aws-assumerole-with-cert

Wed, 26 Jan 2022 03:27:23 +0000

AWS AssumeRole with Certificate for CI

Exploring the concept of using AWS IoT Certificates for authenticating with AWS.

This came up while working with minio, which supports authentication with certificates:

MinIO provides a custom STS API that allows authentication with client X.509 / TLS certificates.

cloudfront-cognito-private-auth

Wed, 26 Jan 2022 02:50:02 +0000

AWSS3 & AWS Cognito

Fiddling with AWS S3 Websites leveraging AWS Cognito for authentication

Terraform is based on the tutorial https://transcend.io/blog/restrict-access-to-internal-websites-with-beyondcorp/ and the public repository: https://github.com/transcend-io/beyondcorp-cloudfront

Notes

This approach isn’t really something I think is great
The lack of ease configuration for Lambdas, and the need to either embed configuration in the lambda zip, or through AWS SSM is not ideal
Either a single monolith terraform module, or split between multiple. Adds a lot of overhead which is less than ideal
AWS S3 just doesn’t feel like a good source for storing the S3 artifacts - requires bucket policy + “public” - Don’t like
Seems more preferable to have something like K8s + Nginx pod - Management of secrets/dns/etc can be externalized then
Docker images gives the opportunity to sign with something like cosign, ability to “extend” with URL rewrites/etc through nginx itself
Authentication can be fronted as part of existing K8s services used by backends
Entire system can be validated with a local installation of K8s
Alternative caching layers seem more ideal (e.g. Cloudflare pages / etc), thing that can go infront of the primary service
Could a k8s-ingress served entity just synchronize with other CDNs?
Cloudfront “deploying” wait times are kind of in the way for rapid changes (exists in k8s with pods, but cloudfront just feels slower?)

Overall not really liking this approach, as it doesn’t simplify the system as much as I was hoping it would.

rules_docker

Wed, 12 Jan 2022 01:24:06 +0000

Bazel Container Image Rules

Bazel CI

Generated API documentation is in the docs folder, or you can browse it online at https://docs.aspect.dev/rules_docker

Basic Rules

These rules used to be docker_build, docker_push, etc. and the aliases for these (mostly) legacy names still exist largely for backwards-compatibility. We also have early-stage oci_image, oci_push, etc. aliases for folks that enjoy the consistency of a consistent rule prefix. The only place the format-specific names currently do any more than alias things is in foo_push, where they also specify the appropriate format as which to publish the image.

dapper-with-entity-model

Sat, 23 Oct 2021 14:43:17 +0000

EntityModel Dapper Experiment Case

Exported case of experimenting with using Postgres Functions, Dapper & Entity.Model.

Notes

Experimenting a bit more with the entity model of splitting out the “database” components (e.g. ID) from the model objects, with an aim towards designing a strict interface for working with the database, that could be generated from a baseline specification. This would allow for things like ReadOnly entities, better field filtering & potential for “programmatic” improvements to how the database handles searches.

cue-for-schema-gen

Sat, 23 Oct 2021 00:46:21 +0000

Cuelang with SchemaGen

Experimenting with using Cuelang for the purposes of representing a schema, then generating associated files from the original source of truth

Notes

Schema validation is nice, the base case is straightforward
Combining data with this allows for connecting enum/datasets
What about stubbing of datasets (e.g. restrict this to ‘Dataset’ that isn’t locally defined?)
Text templating isn’t really what I want to do with this kind of tool
Seems like I need to write more “representation/composition” than I originally hoped
Doesn’t seem to support the kind of “intentions” workflow I was hoping for

This isn’t handling the case I’m interested in with schema validation & generation. Although it provides a lot of the base essentials out of the box, I’m looking more for a cross between schema+terraform. A system that allows the tracking of version drift, and reconcilation as part of its design.

bazel-bash-packaged

Tue, 05 Oct 2021 23:00:41 +0000

Bazel Bash Packaged

Experimenting with using Bazel & Bats in container images for writing up tests for shell scripts.

Notes

Sometimes while developing things it can be useful to have a small shell script that performs a simple action or operation, that in the short-term makes sense as a shell script. In this case, it can be useful to add some simple tests to ensure that:

jcompiler

Mon, 04 Oct 2021 23:31:06 +0000

JCompiler

Summary

A Joos programming language compiler, written in Java.

Getting Started

The project is currently not maintained or kept in runnable order. You may be able to open the project in Eclipse, but at this time the code is only here as readonly.

bazel-toolchain-from-s3

Sun, 26 Sep 2021 23:53:42 +0000

bazel-toolchain-from-s3

Experimenting with setting up Bazek toolchains, when the tools are mirrored into an AWS S3 bucket.

This builds off previous work done in jrbeverly/bazel-external-toolchain-rule for creating toolchains from files.

Notes

Implementation of s3_archive uses that same model as http_archive
The repository_rule does not use toolchains like rules, meaning bootstrap rules must be downloaded by repository_ctx
Repository rules can convert labels to paths using the repository_ctx.path method

To get a fully managed system that doesn’t require tools to be installed on the local system, would likely require a bootstrap rule using a tool that is publicly available. This way it could be downloaded using the download_and_extract rule of Bazel, then combined with the other rules for perform the act of downloading binaries. The process as follows:

bazel-external-toolchain-rules

Sun, 26 Sep 2021 22:20:07 +0000

bazel-external-toolchain-rules

Experimenting with setting up Bazel toolchains using an externally managed .toolchain file, that is responsible for defining properties such as:

System compatibility
Integrity Checks
Tool retrieval locations

Notes

This idea came out of the idea of having Bazel rules that were not aware of how toolchains were defined (or what systems they are compatible with), and instead being entirely based on the lock file (.toolchain) available in the caller environment. This means that the bazel rules can in essence just be extensions of the command line specification.

bazel-jsonnett-templates

Sun, 12 Sep 2021 17:15:18 +0000

Bazel & Jsonnet Templates

Generating files from base configuration files using Jsonnet.

Notes

Exploring the idea of leveraging jsonnet with Bazel to create a series of templates sourced from configuration files. The basic principle of this is how to built in-repository the idea of ∀ ∈ Configuration Files { template([inputs]) => rendered }.

internal-reserved-license-repo

Sat, 11 Sep 2021 19:18:11 +0000

internal-reserved-license-repo

Experimenting with laying out the licensing stamp for a closed/internal source repository

Notes

Experimenting with the idea of what license annotations would look like on an internal repository that is not intended for public distribution. This can seem odd, as the source code files of the project should not be distributed, so the only individuals viewing the license should be those with pre-approved access (i.e. contributors/employees).

github-actions-dbx-upload

Sat, 28 Aug 2021 22:26:52 +0000

github-actions-dbx-upload

Publishing to Dropbox programmatically from GitHub Actions with the intentions to mirror the model of AWS S3 publishing.

Notes

Exploring how one might leverage Dropbox as an artifact storage source, with an authentication model that is similar to AWS. Setting up the initial Dropbox Application to get the tokens for publishing was a bit more involved than useful, and I suspect the process of rotating these tokens might be a pain too.

terraform-aws-codepipeline-terraform-runner

Sat, 28 Aug 2021 14:39:46 +0000

Terraform AWS CodePipline Terraform Executor

Terraform executor leveraging the CodePipeline functionality in AWS, for a fully serverless model of executing terraform in AWS.

Getting Started

The primary environment is configured in env/, with a single main file. This provisions the CodePipeline, CodeBuild components, as well as some stub S3 buckets that represent both incoming sources, artifacts. The artifacts bucket is treated as a stand-in for the Terraform state environment variable, but the pipeline itself does not attempt to configure the backend for S3.

golang-analyzer-inline-bazel

Wed, 18 Aug 2021 01:16:50 +0000

Bazel Golang Inline Analyzer

Experimenting with having analyzers locally defined to a repository, rather than externally defined.

Notes

Requires using go_tool_library instead of go_library due to a dependency change issue (must also use go_tool_library of deps)
Baked natively into nogo, so it can be pretty straightforward to test
Names of types aren’t as simple as package.Type, but instead include other components (using HasSuffix) (What options are there?)
Change in ‘internal/cobrago/storage.go@ListFilesInStorage’ can be removed as a test case for the errors
The tools/ directory probably isn’t the best path. Want something that we can spin-out/externalize as these evolve with the code

Overall this is a pretty good way to start prototyping mechanisms for code analysis that is right next to the code, which can then later be spun out into their own generic analyzers. As it runs automatically with nogo, there is a natural way of enabling an analyzer in a tiered manner (warning => error).

github-app-golang

Fri, 16 Jul 2021 16:57:05 +0000

GitHub App in Golang Prototyping

GitHub App written in Golang with the AWS & GitHub integrations split away, to try and encode the core ‘concepts’ solely into the lib/ component

Notes

Development (& Testing) should support a non-smee way of development
How would integration/infrastructure/e2e tests work with this kind of system?
Would it make more sense to have an OpenAPI system, with the GitHub integration performing the interface?
Is this similar to Hubot, in that having an interface layer/service makes more sense, as it avoids the requirement for direct interface? (This can be within the system itself)

k3s-at-home-poc

Mon, 12 Jul 2021 20:40:29 +0000

K3s In HomeLab Proof of Concept

Determining how viable it would to be switch from using docker-compose to using K3s to run my internal homelab environment.

Getting Started

The installation process assumes that you have a freshly imaged ubuntu machine, connected to the internal network, and with a minimum of password-based SSH access.

gitpod-cobra-golang

Mon, 12 Jul 2021 19:10:27 +0000

GitPod Golang CLI

Leveraging GitPod for prototyping out a golang cli that interfaces with AWS.

Notes

The .gitpod.yml file must exist in the root directory
Dockerfile(s) for the environment can be specified in its own directory (.gitpod)
Commands can be run on start-up, ensuring that the build is working as expected
GitHub Permissions required to make changes to GitHub Actions workflows
GitHub Permissions required for a series of commit/pull request based actions
Workspaces can be provisioned/stopped/cleaned up on-demand
Docker works on the provisioned nodes
Extensions & Other components are defined by the .gitpod.yml file

In comparison, the code-server approach is to bake all the dependencies into the same image that is running code-server itself. This means that extensions/settings/etc can be baked onto the image that are common across projects. If working in Docker, it requires a bit more process to work with other docker processes (running on same docker network, matching file system paths, etc).

rust-lang-checks

Mon, 12 Jul 2021 15:05:14 +0000

Rust Language Checks

Experimenting with aspects of Rustlang for working with database, and immutable data structures.

Notes

How might the Entity.Model apply to this? Where we split the id from the data struct?
Immutable hashmap appears to need additional crates (cursory look)
The /src & /lib standard directories is a nice component
Using .env to reference the database URL
Requires some additional components for PSQL & Diesel

packer-bake-with-aws-native

Sat, 01 May 2021 01:10:12 +0000

Packer Bakery with AWS Native

Creating pre-baked AMIs using Packer within AWS Native resources (Codepipeline / CodeBuild).

Notes

Tuning minimum permissions can be a bit difficult with the CodePipeline/CodeBuild error messages
Artifacts bucket should store only temporary/cache files, and should be destroyable
Logs from CodePipeline & CodeBuild can be restricted to a specific log group
Unique ‘key’ identifier allow single-use of module within a common key-scope
Events on-complete require additional resources/overhead
Image is amazon pre-built, installing Packer on-the-fly

Although nice to leverage IAM solely for this, the benefits don’t really outway the issues with leveraging CodePipeline for this kind of build. Splitting this CI/artifact process is less than ideal, but granting credentials to external providers that can spin up any EC2 & run arbitrary scripts has its concern points.

bazel-and-aws-cdk

Tue, 27 Apr 2021 23:00:39 +0000

bazel-and-aws-cdk

Prototyping ideas with using Bazel and AWS Cloud Development Kit to create cloudformation templates

repository-template-file-invoke-prototype

Thu, 25 Mar 2021 21:47:15 +0000

Repository Templating & File Automation

Experimenting with a model of building a lightweight cron+bash system for performing templating&file modification to multiple repositories.

Idea

The basics of this repository was the idea of leveraging the GitHub CLI (gh) to automate the creation of pull requests that were intended to facilitate common chore work in repositories. This would reduce the need to handle things like setting up license files, formatting, metadata, GitHub Actions, etc.

dotfiles

Sun, 21 Mar 2021 02:26:13 +0000

dotfiles

A set of vim, zsh, git, and configuration files.

ml-learning-lab

Tue, 16 Mar 2021 02:03:08 +0000

Machine Learning Lab

A repository for aggregating my machine learning exercises, practices and learning labs. The projects included in this repository are based on the coursework for Udacity’s Deep Learning Nanodegree Foundations. These are primarily from working on the Machine Learning Nanodegree offered by Udacity.

codespace

Mon, 08 Mar 2021 01:03:04 +0000

Codespace

Prebuilt, development environment in the browser – powered by VS Code.

This image acts as a catch-all image for doing full-stack development in a polyglot type environment. The running container makes use of the host docker service to allow for docker builds.

home

Sat, 28 Nov 2020 02:46:39 +0000

HomeLab - Internal Tooling

Ansible playbooks for configuring services running within my internal home cloud.

Getting Started

DevContainers

The DevContainer environment can be started by opening the repository in VSCode and installing the ‘Remote - Containers’ extension. When started, the prompt will build and image and configure the container.

aws-lightsail-codespaces

Thu, 20 Aug 2020 22:21:48 +0000

Running code-server on AWS Lightsail

Summary

Run VS Code on an AWS Lightsail instance with auto-generated password and static IP. Early experiments with cloud-driven development environments configured on-demand using terraform.

Initial exploratory work for seeing what changes exist in the workflows, and any issues that may arise as a result of working in Lightsail.

aws-lambda-simple-service

Wed, 19 Aug 2020 23:16:36 +0000

Express in Deployments

A simple Express application built with the intent to test an Express server running in different environments (local, docker, lambda).

Usage

The available make commands can be listed with make help:

jrbeverly.web

Sat, 27 Jun 2020 21:01:56 +0000

jrbeverly

Represents the infrastructure resources of ‘jrbeverly’, keeping track of infrastructure components, assets and other resources that are needed for components.

Directory Mappings

The directory layout is modelled after the Linux directories (/var, /etc/, /root, /srv, /opt, etc). This is intended to handle cases as they are adopted into the standards.

boston-housing

Mon, 24 Feb 2020 17:50:55 +0000

Predicting Boston Housing Prices

Evaluate the performance and predictive power of a model that has been trained and tested on data collected from homes in suburbs of Boston, Massachusetts. A model trained on this data that is seen as a good fit could then be used to make certain predictions about a home — in particular, its monetary value. This model would prove to be invaluable for someone like a real estate agent who could make use of such information on a daily basis.

charityml

Mon, 24 Feb 2020 17:50:35 +0000

Finding Donors for CharityML

Employ several supervised algorithms to accurately model individuals’ income using data collected from the 1994 U.S. Census. From the best candidate algorithm from preliminary results and further optimize this algorithm to best model the data. Construct a model that accurately predicts whether an individual makes more than $50,000. This sort of task can arise in a non-profit setting, where organizations survive on donations. Understanding an individual’s income can help a non-profit better understand how large of a donation to request, or whether or not they should reach out to begin with. While it can be difficult to determine an individual’s general income bracket directly from public sources, we can (as we will see) infer this value from other publically available features.

customer-segments

Mon, 24 Feb 2020 17:48:47 +0000

Creating Customer Segments

Analyze a dataset containing data on various customers’ annual spending amounts (reported in monetary units) of diverse product categories for internal structure. One goal of this project is to best describe the variation in the different types of customers that a wholesale distributor interacts with. Doing so would equip the distributor with insight into how to best structure their delivery service to meet the needs of each customer.

dog-project

Sat, 11 Jan 2020 18:05:28 +0000

Classification of Dogs

My implementation of the Convolutional Neural Networks (CNN) algorithm for identifying a canine’s breed from an image. Additionally, it supply the resembled dog breed if provided an image of a human.

quadcopter

Fri, 10 Jan 2020 02:31:05 +0000

Quadcopter using Reinforcement Learning

My implementation of the DDPG reinforcement learning algorithm to solve the problem of a quadcopter taking flight.

I have included a reference to the DDPG paper used in the development of the flying agent:

bmx

Wed, 08 Jan 2020 18:15:28 +0000

BMX

BMX grants you API access to your AWS accounts, based on Okta credentials that you already own.
It uses your Okta identity to create short-term AWS STS tokens, as an alternative to long-term IAM access keys. BMX manages your STS tokens with the following commands:

git-timeline

Sat, 04 Jan 2020 01:26:48 +0000

Git Timeline

Allows bulk modification of the commit dates of a repository, changing the history of a repository.

Usage

# Creates the demo repository
./git-timeline.bash clone

# Copies the demo repository to the working environment
./git-timeline.bash working

# Exports the history of the git repository to files
./git-timeline.bash history

You can then edit the dates of the three files emitted:

aws-lambda-remote-session

Fri, 03 Jan 2020 22:26:00 +0000

AWS Lambda PowerShell Example

A simple Lambda function written in PowerShell to validate if New-PSSession can be leveraged from an AWS Lambda.

The objective of this repository was to determine if it was possible to use New-PSSession from an AWS Lambda. This would allow for modification of services like Office365 using remote sessions from AWS without requiring an ECS/EC2 container. Without a custom lambda runtime (or some way of enabling WSMan), it would be difficult to do this with a vanilla lambda execution environment. The error encountered during runtime is included below:

aws-chat-app

Mon, 30 Dec 2019 23:57:22 +0000

AWS Chat App

SAM Application for a simple chat application based on API Gateways new WebSocket API feature. This was originally developed as an experiment to see how viable running a chat-bot in a fully serverless environment, as opposed to just running on a container in ECS.

hubot-in-aws

Sat, 28 Dec 2019 15:56:31 +0000

Hubot in AWS ECS

Hubot deployment in AWS using AWS ECS Fargate. This was prototyped out while I was evaluating ChatOps strategies that could be used to wrap existing web interfaces or require minimal overhead.

pwsh-from-github

Sun, 17 Nov 2019 15:13:06 +0000

Powershell Library on GitHub

Summary

A powershell library that is installed from GitHub, rather than from Powershellgallery.

Usage

Downloading from GitHub:

# Enable installing from github
Install-Module -Name InstallModuleFromGitHub

# Install the module
Install-ModuleFromGitHub -GitHubRepo jrbeverly/pwsh-from-github

# Perform Actions
Write-Hello -Name "World"
Write-World -Message "LFG"

Notes

Experimenting with using InstallModuleFromGitHub, instead of using Powershell gallery.

proposals-concept

Wed, 30 Oct 2019 03:40:50 +0000

Proposals

Experimenting with the underlying infrastructure for a GitHub based proposals mechanisms that deploys to a web resource (website/subpage/etc)

Motivations

Automation to ensure contribution guidelines are both accessible and followed (e.g. first contribution bot, linting)
Contribution previews, either by links to markdown rendering or branch previews
Support for supplemental resources (e.g. assets/list-of-items.csv)
Methods to organize the proposals better (date? slugs? IDs?)
Simple layout in code (minimal overhead), with tooling to move assets into the correct locations
When working with code, the proposals have some organization mechanism
Minimal barriers when trying to write a proposal

Notes

Using date-based directory organization helps for filtering based on the latest proposals. If you are looking for the proposals in the most recent year, its fairly easy to complete that query (ls designs/2019/). I find that this falls apart when navigating the history for proposals related to a project. I could enforce directory naming standards, but I feel that makes the repository difficult to approach. If I want all the proposals related to a certain project, I’d need to use tools like jq or grep.

exp-circleci-orbs

Wed, 30 Oct 2019 02:21:11 +0000

Experimenting with CircleCI Orbs

Experimenting with CircleCI Orbs for reducing code re-use in templates

Usages

cp Deploy to S3

orbs:
 awscli: jrbeverly/awscli@0.0.5
version: 2.1
workflows:
 Deploy to S3:
 jobs:
 - awscli/aws-copy-to:
 bucket: hello-bucket
 namespace: some/terraform
 source: some/path

sync Deploy to S3

bazel-csharp-testcases

Sat, 26 Oct 2019 21:27:51 +0000

Bazel CSharp Rules Examples

Overview

This repository provides a set of usages for the bazel csharp rules. The idea behind these examples is to cover edge cases that are encountered during development, and provide a comprehensive test (& prototype) suite.

viewdocs-autodoc

Sat, 05 Oct 2019 20:47:31 +0000

Viewdocs Autodoc

This was an early concept I was working with for converting metadata files (json/yml) into standard README markdown files using simple bash and templates. Ultimately I did not go in this direction, as I found that I was not making the best use of the customization yielded from using with bash + templates. A simple templating engine provided all the basics that I was ultimately after.

awesome-terraform-prototype

Sat, 05 Oct 2019 20:41:57 +0000

Awesome Terraform Prototype

An experiment using mkdocs and a series of json/yml files to define an awesome list.

The markdown files are automatically generated from the yml files that define each element of the list (tags/metadata/etc). These files are then piped into mkdocs, which yields a material theme website for the project.

exp-svg-icon-processing

Sat, 05 Oct 2019 20:37:56 +0000

SVG Icon Processing

Summary

Experiment with programmatically generating color variants for SVG files using a JSON definition file with the source SVG.

Usage

The variants of each of the files is defined as a dictionary (string:object). The key for the dictionary matches the name of the variant. The object defines a collection of id and properties. These will be merged into the SVG to generate the variant icon. An example definition file is included below.

bullseye-exp

Sat, 05 Oct 2019 20:35:47 +0000

BullsEye Experimentation

Summary

Experiment with BullsEye for building command-driven tooling (build-systems).

Usage

Experimenting with using BullsEye in a dotnet project. BullsEye doesn’t handle parsing of command line arguments, instead recommends using a tool for parsing them.

lab-starter

Sun, 01 Sep 2019 20:31:22 +0000

GitHub Learning Lab - Lab Starter

Noticed this when working with GitHub Actions that you can have a automated ’teacher’ by using linting + GitHub Bot. Thought this was an interesting idea, and have started this repository to get a better understanding of how this actually works.

hello-github-actions

Sun, 01 Sep 2019 19:56:26 +0000

Welcome to “Hello World” with GitHub Actions

This course will walk you through writing your first action and using it with a workflow file.

Ready to get started? Navigate to the first issue.

dotnet-native-corert

Sun, 03 Feb 2019 03:52:04 +0000

.NET Core Native Compilation Experiments

A simple CI/CD pipeline making use of CoreRT to build linux and windows copies of a “Hello World” console application.

Notes

CoreRT currently does not support compilation cross-compilation (as it is not supported yet).
AOT for a linux binary requires non-dotnet dependencies (that need to be installed to be base image)

At present it is likely best to use Windows docker images for building the dotnet native jobs.

make-exp

Wed, 12 Dec 2018 22:46:15 +0000

Makefile Experiments

Summary

Experimenting with using makefiles as a build harness type structure. The idea is to package makefile using GitHub, that can then be downloaded when running.

Conceptual Usage

As the structure is simply an experiment, no targets are actually implemented. At the top of the Makefile, you can include the makefile using the following:

infrastructure-labs

Fri, 02 Nov 2018 23:41:32 +0000

Infrastructure

Summary

The specification of jrbeverlylabs as a set of terraform modules.

Usage

To run this you need to execute:

terraform init
terraform plan
terraform apply

Notes

This was a simple experiment making use of the gitlab provider of terraform. The idea was to see if it would assist in the process of maintaining jrbeverlylabs between gitlab.com and my internal gitlab instance.

xunit-metadata

Fri, 02 Nov 2018 00:52:39 +0000

XUnit.Metadata

Summary

Strongly-typed attributes for the management and organization of tests. As opposed to using strings throughout the code, [Trait("Category", "Unit")], you can use strongly-typed attributes for organizing tests.

Getting Started

With xUnit v2 you can markup tests with traits, particullary of interest is the category key. Using traits you can sort tests or control execution of tests from the command line. You can install in your project using the Nuget Manager Console:

wifi-web

Fri, 02 Nov 2018 00:52:35 +0000

Wifi Web

Summary

Wifi Web provides an autorun USB for connecting to wireless access points for devices that do not have access to a camera. It opens an HTML page that provides easy access to the Wifi connection details.

stack-opengl

Fri, 02 Nov 2018 00:52:30 +0000

stack-opengl

Summary

stack-opengl is a variant of stack-net written in OpenGL. It uses extremely simple shaders and OpenGL programming to create a block stacking application.

Getting Started

The project uses premake4 as the cross-platform build system. You will need to build the external dependencies of the project, by running a root level build. You can then build the project itself. You can do so as such:

stack-net

Fri, 02 Nov 2018 00:52:29 +0000

Stack-NET

Summary

A block blueprinter, built using a visual graph style approach to graphics.

Getting Started

The project is based on the old approach to C# projects. The project should be opened in Visual Studio, built, then run.

raytracer

Fri, 02 Nov 2018 00:52:27 +0000

RayTracer

Summary

A Raytracer that receives a scene defined in lua, and produces an image output.

Getting Started

Compilation follows the standard process defined by the UWaterloo CS488 sample projects.

We use premake4 as our cross-platform build system. First you will need to build all the static libraries that the projects depend on. To build the libraries, open up a terminal, and cd to the top level of the CS488 project directory and then run the following:

profile

Fri, 02 Nov 2018 00:52:26 +0000

jrbeverly.profile

Summary

This is a one page user profile for Jonathan Beverly (jrbeverly - i.e. me), linking to multiple online identities, relevant external sites, and popular social networking websites. Not all of them are included, but most of the relevant ones are.

exp-portfolio

Fri, 02 Nov 2018 00:52:25 +0000

jrbeverly portfolio

Summary

Collections data from a specified list of gitlab projects, then converts them into static HTML briefs.

Getting Started

The project is designed to git clone a series of repository, then collect information from each of them. This includes the project icon, name, license, path, etc.

office-depot

Fri, 02 Nov 2018 00:52:22 +0000

office-depot

Summary

office-depot is a container based software development stack.

Getting Started

Getting started is as simple as using docker-compose. You can do so as such:

docker-compose up --env-file=office-depot.env -d

Updating and Upgrading

If you wish to upgrade the container stack, you need to run the following commands:

mirroring

Fri, 02 Nov 2018 00:52:21 +0000

Mirroring

Summary

A lightweight bash script that allows easy mirroring of projects to external git hosts.

Getting started

Simply fork this repository, as it has all the scripts necessary for performing mirrors. You can then add your repositories into the assets/ directory. You will want to store them as such:

localization-net

Fri, 02 Nov 2018 00:52:18 +0000

Localization.NET Concept

Summary

A simple experiment prototyping a concept for strongly typed language terms.

Note: The generated component is not built with this. This is a usage prototype only (no generator is included).

issues-style

Fri, 02 Nov 2018 00:52:13 +0000

Issues.Style

Summary

A style guide for issue management, release versioning, Git Flow and repository documentation.

Description

In order to speed up the initialization process of a new gitlab project, Issues.Style provides a set of common labels and issues that might be used when setting up a new project. The project provides methods for quickly setting up the project, specifically providing the following:

project-icons

Fri, 02 Nov 2018 00:52:12 +0000

jrbeverly.icons

Summary

A collection of scalable vector graphics (SVG) that define project and group icons.

Build

You can build the icons using the tool rsvg-convert. To build with rsvg-convert, you can do the following:

gitlab-ci-yml

Fri, 02 Nov 2018 00:52:10 +0000

gitlab-ci.yml

A collection of GitLab CI configuration files that are used by my projects. Stored here as the process of docker projects are polished and standardized.

Getting Started

Each of the dockerfiles is presented with a simple .gitlab-ci.yml file that uses one of my docker images. The resources referenced by the definition are not included in this project. You can start by copying the .gitlab-cy.yml, then replacing the relevant bits.

entity-net

Fri, 02 Nov 2018 00:52:07 +0000

Entity.NET Concept

Summary

A simple experiment prototyping a concept for strongly typed ORMs.

Getting Started

The idea that Entity.NET is attempting to conceptualize is one where strongly-typed objects are used with an ORM system. The primary objective is to use strongly typed identifiers (Keys.Customer) that restricts the usages of an applications ORM. The concept is from the following scenario:

distributedrpc

Fri, 02 Nov 2018 00:41:09 +0000

DistributedRPC

Summary

A multi-client, multi-server environment that relies on a binder to facilitate an RPC system.

Getting Started

To make (“compile and link”) all components of the project, you can quickly get started with

contentbundler

Fri, 02 Nov 2018 00:41:04 +0000

ContentBundler

Summary

A proof of concept for generation of strongly typed paths using the Roslyn Framework.

Getting Started

ContentBundler is provided as an command line application, originally adapted from an XNA Content Compiler. The new version greatly simplifies the code requirements, leveraging Roslyn for the code generation. An example is available from test/assets , which will generate the result [PatchQ.cs]

udacicards

Sat, 07 Apr 2018 20:55:53 +0000

UdaciCards

Summary

For the UdaciCards project, you will build a mobile application (Android or iOS - or both) that allows users to study collections of flashcards. The app will allow users to create different categories of flashcards called “decks”, add flashcards to those decks, then take quizzes on those decks.

jollybot

Sat, 07 Apr 2018 20:50:44 +0000

JollyBot

Summary

Prisoners’ Dilemma A.I. bot performing an ‘Olive Branch’ strategy focusing on attempting to cooperate whenever possible. The bot attempts to establish cooperation, even in cases where the opposing agent may appear hostile (e.g. always defect).

readable

Wed, 17 Jan 2018 23:40:48 +0000

Readable

Udacity Nanodegree React Project

Installing Server

cd src/api
npm install
node start

Installing Client

cd src/client
npm install
npm start

API Server

Information about the API server and how to use it can be found in its README file .

myreads

Sat, 18 Nov 2017 22:49:02 +0000

MyReads Project

Summary

A digital bookshelf app that allows you to select and categorize books you have read, are currently reading, or want to read.

Description

MyReads is a digital bookshelf app that allows you to select and categorize books you have read, are currently reading, or want to read. It is built as a project for Udacity React Nanodegree.

githooks

Sat, 16 Sep 2017 01:04:05 +0000

Githooks

Summary

GitHooks provides a multi-hook framework for Git Hooks, along with a collection of scripts for the purposes of encouraging a commit policy, altering the project environment depending on the state of the repository, and implementing continuous integration workflows. The framework allows multi-script execution, you can use GitHooks to automate or optimize virtually any aspect of your development workflow.

homelab

Wed, 31 May 2017 22:54:15 +0000

Homelab

Summary

A collection of templates and utility scripts used in my homelab. Most of these are just snippets or experiments.

Getting Started

As most of the scripts are self-contained, you can clone the repository:

jotto

Wed, 31 May 2017 14:21:28 +0000

Jotto

Summary

Jotto is a logic-oriented word game played with two players. Each player picks a secret word of five letters (that is in the dictionary), and the object of the game is to correctly guess the other player’s word first. Players take turns guessing and giving the number of Jots, or the number of letters that are in both the guessed word and the secret word.