Today I Learned

Force push safely using --force-with-lease

We periodically force push feature branches when we rebase them.

To do this safely, I first check manually to see if my pair has pushed any commits I don't yet have in my local branch.

But there's an easier and safer way to do this. The --force-with-lease option will fail if there are upstream changes not yet merged.

It will work smoothly for standard rebasing situations, but protect against any inadvertent destructive force pushes.

Thanks to Jason K for sharing this tip with me!

Finding Where a Method is Defined

One great thing about Ruby is how flexible it is. Although sometimes it can be hard to determine where a method definition comes from.

One useful technique to pull out in just this circumstance is the source_location method.

For example, suppose you see a call to the quantity method in a spec and you wonder where its definition comes from. Do some puts debugging by adding this line:

puts method(:quantity).source_location

Lambda argument passing shorthand

Suppose that I have a Ruby lambda like so:

greeting = -> name { "Hello #{name}" }

The conventional way of calling this lambda is like so:

greeting.call("Ned") # "Hello Ned"

However, for those of you dreading having to type in call (hello, JavaScript developers), you can use this syntactic sugar instead:

greeting["Ned"]
greeting.("Ned")

Source: https://stackoverflow.com/questions/18774139/how-works-with-lambdas

UPDATE

I am now a sad panda to learn that this shorthand syntax is not recommended in the ruby-style-guide: https://github.com/bbatsov/ruby-style-guide/issues/205 https://github.com/bbatsov/ruby-style-guide/blob/master/README.md#proc-call

It's still a cool learning for me to know that this syntax exists in the first place, though.

FactoryGirl identifiers as symbols not strings

Why is FactoryGirl.create(:my_object) better than FactoryGirl.create("my_object")?

Symbols are immutable objects that are stored by the Ruby interpreter as a numeric ID (an implementation detail, the design of Ruby is flexible for different storage mechanisms).

Strings, on the other hand, take up a memory footprint as large as the number of characters in the string. Using symbols will thus take up less memory and potentially be faster because the same address in memory will be used.

If you have n instances of the same string in your code, Ruby will use O(n) memory to store all those strings, however, if you have n instances of the same symbol in your code, Ruby will use O(1) memory to do the same lookup.

Further reading: https://bugs.ruby-lang.org/issues/7792#note-58

This TIL came courtesy of a discussion with Evan and later on Arturo about best practices in writing test code, but shared here because it's broadly applicable and the FactoryGirl example is just one use case.

Z-Index Equivalent in SVG

To mimic the z-index CSS property in SVG, add the SVG elements to the page in the reverse order that you expect their z-positioning to be set. The subsequent elements added are placed on top of previous elements.

For example, if I were using the React SVG charting library Victory and I wanted my chart lines to be stacked on top of my chart area shading, I will add them to my component like so:

<VictoryChart>
  <VictoryArea ... />
  <VictoryLine ... />
</VictoryChart>

Combine all ES6 generator values into an array

The ES6 spread operator can be applied to an iterable to combine all its results into one single array. A trivial use case for this is to merge extra elements into an existing array (ie, [...existingArray, 2, 3]).

This usage of the spread can be applied to a Generator to combine all the results of the calls to next() into one array. This serves as syntactic sugar to replace needing to use a for-of loop to do the same operation. Here's an example (illustrative purposes only, ignore that this is not the best way to solve this problem):

function* evenNumbersGenerator(maxNum) {
  for (let i = 0; i <= maxNum; i += 1) {
    if (i % 2 == 0) {
      yield i;
    }
  }
}

...

const evenNumbersTo100 = [...evenNumbersGenerator(100)];

Side Learning: Generator functions cannot be declared as an arrow function, because they have to be declared as a non-method function.

Surround <g> around array of React SVG components

It is a typical practice when writing a React component that renders a list/array of components to surround these components around a <div> tag. This is because React does not directly support rendering arrays of components.

This practice will not work if the React components will deep-render SVG elements instead of HTML markup. The solution instead will be to wrap the array of SVG-based child components around <g> tags (which stands for group in SVG).

https://www.smashingmagazine.com/2015/12/generating-svg-with-react/

Exclusion using File Masks

When using Find in Path.. functionality to find a word across multiple files, file masks can be used to match certain types of files. For example, using *.rb will only look through Ruby files. On the other hand, if you want to exclude Ruby files instead, you can add an ! at the front (i.e. !*.rb).

Here are other useful examples/combinations:

all files excluding spec tests: !*spec.rb

all Ruby files excluding spec tests: *.rb,!*spec.rb

Add have_selector() matcher to RSpec Request Specs

If you are used to writing controller specs, you are probably comfortable with the `have_selector matcher. However, in request specs this matcher is not available. By default, you can only do text search inside the request body which leads to brittle assertions.

You can add the have_selector matcher by updating your RSpec config to include the Capybara matchers on request specs as well.

RSpec.configure do |config|

config.include Capybara::RSpecMatchers, type: :request

end

Then you can write more confident Request specs by using assertions like expect(response.body).to have_selector('ul li', text: 'List content here!')

Stop a docker-compose container without removal

Suppose you need to stop your docker-compose container but you don't want to remove the container. For example, this container runs your database, so removing the container would mean that you would also lose all your data.

Don't use docker-compose down. This will remove your container and you will have rebuild your database from scratch. Sad panda.

Instead, use docker-compose stop. You can then restart the container and have all the data back that you had before.

Resources:

Search DOM by selector in Chrome Inspector

Problem

I want to select a DOM element by CSS selector on my React-generated web page for debugging purposes. In this case, I am trying to test out CSS selectors to be used in my Capybara acceptance test.

Solution

Use jQuery NO, DON'T DO THAT!

A neat feature of the Chrome Web Inspector is that I can search an entire page by CSS Selector, without needing to litter my code with debugger statements. Just open up the Elements tab and then press CMD-F (on Mac) to open up a search bar. I can now type whatever CSS Selector that I want and view the results that match. I can quickly test different selectors out, so that I can verify that I am writing my Capybara test accurately.

Source

An Improvement to the Behaviour of the Git Pager

First, capture your current git pager settings (in case you want to go back):

git config --list | grep core.pager

Second, configure your new and improved git pager settings:

git config --global core.pager "less $LESS --tabs=2 -RFX"
--tabs=2: Is just right for displaying Ruby diffs

-R: Repaint the screen, discarding any buffered input
-F: Causes less to exit if a file is less than one screens worth of data
-X: Leave file contents on screen when less exits.

If you want to get even fancier consider diff-so-fancy.

Bulk Upsert through PostgreSQL 9.5+

Warning: This only works for PostgreSQL 9.5+


Given the following:

  1. a users table:

    • id
    • name
    • created_at
    • updated_at
  2. users table only has 1 user with values:

    • id: 1
    • name: Alejandro
    • created_at: '2010-10-10 10:00:00.000000'
    • updated_at: '2010-10-10 10:00:00.000000'

You can upsert using the following SQL query:

INSERT INTO
  users(id, name, created_at, updated_at)
VALUES
  (1, 'Alexander', NOW(), NOW()),
  (2, 'Belle', NOW(), NOW())
ON CONFLICT (id)
  DO UPDATE SET
    name=EXCLUDED.name,
    updated_at=NOW()
;

Results:

  1. User(id=1) will be renamed from Alejandro to Alexander. The updated_at value will be set to current time.
  2. User(id=2) will be inserted to users table with name = Belle, and both created_at and updated_at will be set to current time.

Chrome: Inspecting DOM elements that require focus

Using Chrome Dev Tools, I occasionally want to inspect an element that requires focus - for example, inspecting a selected date in a date selector.

Problem

As soon as I click the dev tools window, the element loses focus and I can no longer inspect in with the desired state.

Undesirable solution

On occasion when I've run into this scenario, I've placed a breakpoint or debugger statement in the JavaScript function that would be called when focus is lost. The problem with this is that it takes time and I need to locate the appropriate place in code.

Better solution

When in the desired state - for example, with the current date selected - pressing F8 will pause JavaScript execution. I can then inspect the DOM in the desired state.

Diff CSVs With Ease

We're all familiar with diff tools like RubyMine, vimdiff, or the good old diff command. These work well for source code but are not optimal for CSVs files, which may have many columns and rows.

I found myself wanting to write CSV-specific diff tool, but I fought that Not Invented Here impulse and Googled first. I found a good Python library called csvdiff.

Imagine we have the following CSV file:

id,name
1,Alan Turing
2,John McCarthy
3,Edger Djikstra

We want to compare it to another file:

id,name
1,Alan Turing
3,Edsger Dijkstra
4,John Von Neumann

We've removed, edited, and added a row. The summary option confirms our changes:

$ csvdiff --style=summary id one.csv two.csv
1 rows removed (33.3%)
1 rows added (33.3%)
1 rows changed (33.3%)

The detailed view gives us all the gory details:

$ csvdiff --style=pretty id one.csv two.csv
{
  "_index": [
    "id"
  ],
  "added": [
    {
      "id": "4",
      "name": "Jon Von Neumann"
    }
  ],
  "changed": [
    {
      "fields": {
        "name": {
          "from": "Edger Djikstra",
          "to": "Edsger Dijkstra"
        }
      },
      "key": [
        "3"
      ]
    }
  ],
  "removed": [
    {
      "id": "2",
      "name": "John McCarthy"
    }
  ]
}

Range prefix to VIM command (from current line)

When I want to execute a command in VIM over a range of lines (for example, search-replace with :s) starting from the current line, I could execute a command like so:

:.,.+10s/foo/bar/g

This reads as search-replace from the current line (.) until 10 lines after. While useful functionality, it does require quite a bit of typing to execute.

Thankfully, there is a neat little shortcut that can be used to reduce some of this type. Simply enter the number of lines that you want to change from the current line and VIM will fill in the rest in the command prompt.

For example, I need to type 11: and VIM will insert :.,.+10 into the command prompt. You can now finish off this command like needed. Do note that the line count includes the current line itself (think of this count as current line plus number of lines after).

Squish Those Strings

While reading some Rails code, I came across a deprecation warning:

ActiveSupport::Deprecation.warn(<<-MESSAGE.squish)
  `redirect_to :back` is deprecated and will be removed from Rails 5.1.
  Please use `redirect_back(fallback_location: fallback_location)` where
  `fallback_location` represents the location to use if the request has
  no HTTP referer information.
MESSAGE

What caught my eye was the squish method on the heredoc. It's common to see methods after heredocs to clean up formatting, but squish is a great method name.

squish removes all leading and trailing whitespace, then replaces all consecutive whitespace with a single space. One application has been cleaning up long string messages that use the line continuation operator. The community style guide says to only use line continuations for concatenating strings, but I think squish is cleaner:

long_string = "a long string " \
              "spanning " \
              "three lines"
# => "a long string spanning three lines"

better_long_string = "a long string
                      squished to a single line
                      without extra spaces or backslashes".squish
# => "a long string squished to a single line without extra spaces or backslashes"

An all-too-common caveat: squish is an extenstion method from active_support.

Using Trigram Indexes to Speed Up LIKE Queries

Queries containing ILIKE '%anything%' result in table scans since they have a leading wildcard. Typical btree indexes can not help improve performance in this case.

Trigrams are 3-character slices of words:

SELECT show_trgm('Ruby');
-- {"  r"," ru","by ",rub,uby}

They are useful as indexes since they can speed up LIKE, ILIKE, ~, and ~* clauses.

Let's query a table of 14K email addresses:

CREATE TABLE emails (email TEXT);

\copy emails FROM 'emails.csv' DELIMITER ',' CSV;

EXPLAIN ANALYZE SELECT * FROM emails WHERE email ILIKE '%ion%';

The query plan shows a table scan:

Seq Scan on emails
  Filter: (email ~~* '%ion%'::text)
  Rows Removed by Filter: 14040

with an average execution time of 15ms. Let's create a trigram index and try again:

-- postgres 9.2+
CREATE EXTENSION IF NOT EXISTS pg_trgm;

CREATE INDEX emails_search_email_idx ON emails USING GIN (email gin_trgm_ops);

EXPLAIN ANALYZE SELECT * FROM emails WHERE email ILIKE '%ion%';

The improved query plan is:

Bitmap Heap Scan on emails
  Recheck Cond: (email ~~* '%ion%'::text)
  Heap Blocks: exact=98
    -> Bitmap Index Scan on emails_search_email_idx
       Index Cond: (email ~~* '%ion%'::text)

and averages 0.8ms. A 19X improvement without updating any SQL queries! YMMV

bundle update --conservative

Let's say you want to update the gem foo. After running bundle update foo you look at your gemfile.lock and find that not only was foo updated, but so were many of its dependencies. You also notice that many of these dependency updates weren't necessary, as the previously installed versions were compatible with the new version of foo.

The default bundle update foo behaviour will unlock and update all dependencies of foo.

If you don't want to update these dependencies unnecessarily, one solution is to add the current versions to your gemfile.lock.

However, an easier way to prevent the updating of shared dependencies is to use bundler's new --conservative flag.

All the different ways to lock your Mac computer

  • Using the keyboard: CTRL-SHIFT-eject
  • Using Alfred: execute the lock command
  • Using a taskbar icon from Keychain:
    • Open the app Keychain Access
    • Preferences --> Enable the option Show keychain status in menu bar
    • A lock-shaped icon will appear in the taskbar, with the option Lock Screen
  • Use Hot Corners:
    • System Preferences --> Desktop & Screen Saver --> Hot Corners button
    • Set one of the corners as Put Display to Sleep
    • Mouse into the direction of that corner (on the monitor closest to that side, for multi-monitor setups)
  • If all else fails, just hold the Power button on your keyboard for a second to put your computer to sleep

async/await Keywords of ES7 for Simpler Promises

ES7 introduces the async/await keywords. They are mostly syntactic sugar on top of Promises. You can write cleaner more readable asynchronous tasks code.

NOTE: They are available in Node 7.6.0 also.

Example:

Here is some code for programmatically creating a coding exercise for a developer candidate that has applied to us. We've improved the readability of it using async/await.

This hard to read test which uses Promises:

it('returns any errors that may have occurred', () => {
  return automator
    .getRepo()
    .then(repo => !repo ? automator.createExercise() : Promise.resolve())
    .then(() => this.room.user.say('alice', `hubot create coding exercise repo for ${candidateEmail}`))
    .then(() => {
      expect(lastBotMessage()).to.contain(`Error creating coding exercise for *${candidateEmail}*:`)
    });
});

Becomes this:

it('returns any errors that may have occurred', async() => {
  let repo = await automator.getRepo();
  if (!repo) {
    await automator.createExercise();
  }

  await this.room.user.say('alice', `hubot create coding exercise repo for ${candidateEmail}`);

  expect(lastBotMessage()).to.contain(`Error creating coding exercise for *${candidateEmail}*:`)
});

Which reads more like an Arrange/Act/Assert style test.

Use PostgreSQL table as queue skipping locked rows

This command only works in PG 9.5

First, are you sure you shouldn't be using some in-memory queue? Or some message broker? Or Redis?

If you are really convinced that you want this sort of behaviour on top of the goo'old PostgreSQL you can achieve a queue-like access pattern by locking and skipping locked rows.

One way to ensure that only one database client can modify a set of rows is to select the rows for update as in

BEGIN;
SELECT * FROM queue ORDER BY priority DESC LIMIT 1 FOR UPDATE;
...

This will block any other UPDATE or SELECT .. FOR UPDATE until the above transaction is finished. Now if you want concurrent database clients to fetch the next row available in the priority list just use:

BEGIN;
SELECT * FROM queue ORDER BY priority DESC LIMIT 1 FOR UPDATE SKIP LOCKED;
...

The above command could be interpreted as select the next row from queue that nobody has yet locked.

Ignore whitespace-only changes in Gitlab MRs

Problem

You are reviewing a Gitlab Merge Request (MR). It involes changes where a Ruby file is moved into or out of particular modules. This means that one or more module ... end constructs are either added or removed around the entire class, resulting in a tabbing-width change on almost every line of the file. This makes reviewing the file in the MR quite difficult, as the reviewer will need to sift through the code diff and separate out actual logic changes from whitespace-only changes.

Solution

By appending ?w=1 to the end of the URL of the MR, Gitlab will ignore all lines that only feature a whitespace change in the code diff. This means that the only changes visible will be ones with a non-whitespace change, thus allowing the reviewer to focus in on actual logic changes and not waste time reading whitespace changes.

Time to poke Gitlab EE support on this issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/1393

Assignment on associations will bypass validation

Important gotcha that assignment to has_many associations will cause an immediate save even if callback validation fails.

u = User.last

u.accounts = []

u.save # returns false because this user cannot have blank accounts

u.reload.accounts # returns empty array

The gotcha here is that save is a no-op really. As soon as u.accounts=[] is called, the data is saved immediately, bypassing validation.

Fixing ZDT Deploys with Monitoring-Driven Develop.

We were debugging a bug in our zero downtime deploy script. We tested the deploy using Chef, Kitchen and Vagrant. The bug was causing an error page to show for less than 3 seconds at the very end of the deploy.

We used Monitoring Driven Development to consistently reproduce the bug (the red phase), and to validate our fix (the green phase). Here were the steps:

Preparation:

  1. kitchen login # ssh to the vagrant box
  2. sudo tail -fn100 /var/log/nginx/go.access.log # to see the response codes from the server. If all responses show 200 code during the deploy, the deploy is working.
  3. Open other terminal window
  4. kitchen login # ssh to the vagrant box
  5. watch -n 1 "curl localhost" # make a request every 1 second

Then we followed this feedback loop

On the host computer:

  1. Make changes to the deploy recipe
  2. kitchen converge # this will run our deploy
  3. Check the output of the logs to see the result of the changes
  4. Repeat the loop until bug is fixed

Replacing num.times.map with Array.new(num)

Problem

I want to create an array containing objects created with an incrementing integer index parameter. For example, an array of hashes containing strings built off of the incrementing number.

Standard Solution

my_array = num.times.map do |index|
  build_hash(index)
end

BUT...rubocop didn't like this:

... C: Performance/TimesMap: Use Array.new with a block instead of .times.map

Improved Solution

The improved solution allows us to build the same array with less chained methods, thus improving readability:

my_array = Array.new(num) do |index|
  build_hash(index)
end

pgcli — a great alternative to psql

Ever wished for a feature-rich, intuitive command-line postgresql client? Look no further! Presenting, pgcli — a result of pouring creativity into features rather than the name of the tool.

Supports:

  • Smart autocompletion for almost anything. Even column names in the select query are properly autocompleted from the table you are selecting from;
  • Multi-line query editing support;
  • SQL Syntax highlighting;
  • CLI parameters compatible with psql params. No need to relearn anything;
  • Vi mode, allowing to edit multi-line queries using some of the vi bindings. There are also Emacs bindings too;
  • Installation as easy as pip install pgcli (and if that fails, fix is as easy as xcode-select --install (usually));

Grab your copy today!

Speeding Up Rake Command Completion

The speed of rake command completion is determined by the startup time of your Rails app.

If your Rails app is slow to startup, the oh-my-zsh rake-fast plugin will make rake command completion tolerable again.

Edit your zsh configuration file:

plugins=(... rake-fast ...)

After refreshing your shell instance issue the following command:

$ rake_refresh

Take a deep breath. Enjoy fast rake command completion.

Record File Handle Usage in OSX

lsof is a helpful tool for looking at what files a process currently has open, however sometimes a process may only access a file for a second and lsof may miss the moment.

For OSX we also have Instruments. This is included with XCode and is pretty straight forward to use:

  • Open Instruments
  • Select File Activity
  • Select the process
  • Hit Record
  • Perform your action
  • Stop Recording

You can also save the log for later analysis.

webpack-merge is a thing and it is beautiful

Let's say you want to redefine a loader from your base webpack config in your production webpack config. Try webpack-merge.smart !

webpack.base.config.js

const config = {
  entry: "./index.js",
  module: {
    rules: [
      {
        test: /\.css?$/,
        exclude: /node_modules/,
        use: [{
          loader: "css-loader",
          options: { sourceMap: true }, // set to true
        }],
      },
      {
        test: /\.css?$/,
        include: /node_modules/,
        use: ["css-loader"],
      },
    ],
  },
};

webpack.production.config.js

const merge = require("webpack-merge");
const baseConfig = require("webpack.base.config");

const productionConfig = {
  module: {
    rules: [{
      test: /\.css?$/,
      exclude: /node_modules/,
      use: [{
        loader: "css-loader",
        options: { sourceMap: false }, // override to false
      }],
    }],
  },
};

module.exports = merge.smart(baseConfig, productionConfig);

Result

const config = {
  entry: "./index.js",
  module: {
    rules: [
      {
        test: /\.css?$/,
        exclude: /node_modules/,
        use: [{
          loader: "css-loader",
          options: { sourceMap: false }, // yep! it's false
        }],
      },
      {
        test: /\.css?$/, // but we didn't touch this rule
        include: /node_modules/,
        use: ["css-loader"],
      },
    ], // and we didn't append anything either!
  },
};

webpack-merge.smart is aware of the shape of a webpack configuration and allows you to update only the rule you want.

Check it out: https://github.com/survivejs/webpack-merge#smart-merging

Mocha: Fail on any console error message

PROBLEM

I want to write a Mocha (JS) test that will fail if there is any warning printed to the console to fail. My use case to necessitate this requirement is that I want to test whether the correct prop type is passed to a React component. PropTypes exists for this purpose, but would only print out a message to the console instead of failing.

SOLUTION

Use the following code, either at the beginning of your test file or a global helper such as specHelper.js, to stub out the implementation of console.error to throw an Error and thus fail the test.

before(() => stub(console, "error", (warning) => {
  throw new Error(warning);
}));

after(() => console.error.restore());

Reference: this gist.

Capybara will skip invisible elements by default

While working on an acceptance test for a date range filter in GO, we were having an issue where Capybara couldn't find an element on the page, even though we could verify it was there. Eventually we realized that the element had an opacity of 0, and that Capybara was passing it over. To illustrate, imagine you have an element with the id #myElement.

CSS:

#myElement { opacity: 0; }

And in your Rails spec:

page.find("#myElement");

The spec will fail, because #myElement can't be found.

Fortunately, there is a visible option that can be set to false so that Capybara doesn't skip the element. So now, changing the line in the spec to:

page.find("#myElement", visible: false);

will cause it to pass.

Encrypt data using psql + keybase

To export any query to a CSV and send it to stdout one can use:

psql -c "\copy (select version()) to stdout csv header"

So you can just replace select version() with any query in the above command and the results will be dumped in your terminal screen. If you have any sensitive data that is not already encrypted you could pipe this results directly to keybase as in:

psql -c "\copy (select version()) to stdout csv header" | keybase encrypt diogob

Where diogob is the recipient of your message (or your own username in case you want to store this file for future use).

SQL's WITH RECURSIVE Query

While optimizing calls to a recursive table, we found a neat SQL solution. It uses a common table expression as a working table to query against iteratively.

Here's an example of using WITH RECURSIVE with a modified nested set example of clothing categories that find all paths through the categories:

CREATE TEMPORARY TABLE categories (id INT, name text, parent_category_id INT);

INSERT INTO categories VALUES
  (1, 'Clothing', null),
  (2, 'Mens''s', 1),
  (3, 'Women''s', 1),
  (4, 'Suits', 2),
  (5, 'Dresses', 3),
  (6, 'Skirts', 3),
  (7, 'Jackets', 4),
  (8, 'Evening Gowns', 5);

WITH RECURSIVE category_hierarchies AS
(SELECT id, parent_category_id, name AS full_path
 FROM categories
 WHERE parent_category_id is NULL

 UNION ALL

 SELECT child_categories.id,
        child_categories.parent_category_id,
        parent_categories.full_path || ' -> ' || child_categories.name as full_path
 FROM categories AS child_categories
 INNER JOIN category_hierarchies AS parent_categories
   ON child_categories.parent_category_id = parent_categories.id
)
SELECT full_path FROM category_hierarchies ORDER BY full_path;

Produces paths through all categories:

  • Clothing
  • Clothing -> Mens's
  • Clothing -> Mens's -> Suits
  • Clothing -> Mens's -> Suits -> Jackets
  • Clothing -> Women's
  • Clothing -> Women's -> Dresses
  • Clothing -> Women's -> Dresses -> Evening Gowns
  • Clothing -> Women's -> Skirts

Read more about WITH RECURSIVE queries

Run last command in BASh/ZSH (they're different)

To run the last executed command in BASH, execute the following:

!!

In ZSH, things are a little different. Using !! will only expand the command into the shell prompt. You would have to press enter again to execute it. Rather, if you want to immediately execute the last command similar to BASH, use this:

r

If you prefer for ZSH behaviour to match that of BASH, then add setopt no_hist_verify to your .zshrc file.