This article will guide you through all the steps necessary for enabling the Flutter Puzzle Hack starter app to be controlled by your phone's gyroscope.

This tutorial is motivated by two main factors:

Firstly, I wanted to make use of devices' physical sensors, which rarely get the spotlight in applications I work with on daily basis.
Secondly, I wanted to challenge myself to add new functionality to the original code while changing it as little as possible. The idea is to illustrate the benefits of a good app architecture and how it enables us to easily add new features without having to understand the details of the existing code.

This article, however, won't go into detail about the following topics:

It will not explain Very Good Ventures' architecture. I suggest poking around the codebase yourself to get a feeling about it. You can also check out Flutter Community's videos that go into detail about this topic.
It will not explain how phone's sensors work.

To follow the content, you should be comfortable using Dart's Streams and know the basics of bloc state management, specifically with Cubits, which we will be using in this challenge.

Although not necessarily required, you'll have a better experience testing the game on a real device, rather than an emulator or a simulator.

Note that this is a step-by-step guide to the final solution. It takes you through the entire journey that I went through, including a few dead ends.

If you just want working code without all the text, you can find in on GitHub, with most of it placed in lib/movement folder.

Setup

Before we get started, you will need to pull the starter source code.

Optionally, you can also edit analysis_options file to match your coding style. I prefer setting my formatter's line length to 120, since I believe it improves readability, especially when it comes to Widget's build methods. For this tutorial I'm also choosing to ignore the warning about documenting public methods and properties, although VGV's documentation of their source code made it way easier to understand their code at first.

# in analysis_options.yaml
linter:
  rules:
    public_member_api_docs: false         
    lines_longer_than_80_chars: false

Add new puzzle mode

If you the run the starter code on your device, you'll see that it has two modes: "Simple" and "Dashatar". Our goal is to add a new puzzle mode right next to them.

First, create a new folder inside lib and call it movement. This folder is going to contain almost all the code needed to add our functionality. We are going to be splitting our functionality into the logic part (which we will place in bloc subfolder) and the (much smaller) UI part, which will live in presentation folder.

Let's first create MovementTheme and MovementLayoutDelegate classes. The theme will use the style of the SimpleTheme, while the delegate will be in charge of drawing our game.

// in lib/movement/presentation/movement_layout_delegate.dart
import 'package:very_good_slide_puzzle/layout/layout.dart';
import 'package:very_good_slide_puzzle/simple/simple.dart';

class MovementTheme extends SimpleTheme {
  const MovementTheme() : super();

  @override
  String get name => 'Movement';

  @override
  PuzzleLayoutDelegate get layoutDelegate => const MovementLayoutDelegate();
}

class MovementLayoutDelegate extends SimplePuzzleLayoutDelegate {
  const MovementLayoutDelegate();
}

The core of this project will live in a class called MovementCubit. I chose to go with a Cubit instead of a Bloc for this tutorial because this class will receive no events from the outside world (except for the sensor data).

Let's create a Cubit that simply emits Strings. We are keeping it simple for now, and we'll start implementing its functionality only once we fully integrate it with the existing app setup.

We need to make the rest of the app aware of the new game mode that we are adding. First we'll add our MovementTheme to the list of initial themes of the PuzzlePage.

Then we'll also make sure that MovementCubit is initialized and available in PuzzleView, so that it can be accessed by our MovementLayoutDelegate.

// in lib/movement/bloc/movement_cubit.dart
import 'package:flutter_bloc/flutter_bloc.dart';

class MovementCubit extends Cubit<String> {
  MovementCubit() : super('Initial state');
}

// in lib/puzzle/view/puzzle_page.dart
...
        BlocProvider(
          create: (context) => ThemeBloc(
            initialThemes: [
              const SimpleTheme(),
              context.read<DashatarThemeBloc>().state.theme,
              const MovementTheme(), // add our theme
            ],
          ),
        ),
...
          child: MultiBlocProvider(
            providers: [
           ...
              BlocProvider(create: (_) => MovementCubit()), // add our cubit
           ...
...

If you now run the app, you should see a new tab called "Movement" that wasn't there before. Not surprisingly, the puzzle board's look and behavior are identical to the "Simple" variant.

Let's change that!

Detecting device rotation

Getting a phone's position in space turned out to be way more complicated that I initially expected. It involves combining data from multiple sensors (using Sensor fusion, and, more specifically, Complementary filter). I ended up creating a package that does the heavy lifting for me, as well as make the code reusable for the next project where I might need this functionality. If you're interested in learning more about its implementation, a post will be available shortly in which I will describe the process.

For this tutorial, we can make use of the package I created. First, add the package dependency:

flutter pub add tilt

Next, we will display the data we're getting from the package on the screen, in order to confirm that the package does what it says on the tin. This means listening to the Stream of device's position in our MovementCubit and emitting the position as state.

Whenever we want to modify the UI, we need to override methods of PuzzleLayoutDelegate. Its boardBuilder method describes what the puzzle board looks like, given the size of the board and the list of tiles that need to be displayed. We will do a very light override of this method by wrapping the default implementation in a BlocBuilder that ensures that the view gets rebuilt every time MovementCubit emits a new state.

// in lib/movement/bloc/movement_cubit.dart
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:tilt/tilt.dart';


class MovementCubit extends Cubit<String> {
  MovementCubit() : super('Initial state') {
    DeviceTilt().stream.listen((event) {
      emit(event.toString());
    });
  }
}

// in lib/movement/presentation/movement_layout_delegate.dart
...
  @override
  Widget boardBuilder(int size, List<Widget> tiles) {
    return BlocBuilder<MovementCubit, String>(
      builder: (context, state) => Column(
        children: [
          super.boardBuilder(size, tiles),
          Text(state),
        ],
      ),
    );
  }
...

DeviceTilt gives us access to Tilt objects, which in turn contain the estimation of angles on the x and y axes (you can also think of them phone's longer and shorter side, respectively).

If you run the app now, you'll see a label describing the device's tilt at the bottom of the screen. 0 degrees on both axes represents a device lying on a flat surface. 90° on x axis means device is standing upright, while -90° is device standing upside-down. On y axis, 90° represents the device lying on its right side, while -90° means its flat on its left side, with the right side standing up.

You'll see that this holds true even if the device is rotated to landscape mode. From the user's perspective, however, the orientation changing makes it quite impractical to play the game. Therefore, I'd suggest locking the device's orientation to portraitUp only (note that iPads do not take app's preferred orientation into account, so the user will have to manually lock it on these devices).

// in lib/movement/bloc/movement_cubit.dart
import 'package:flutter/services.dart';
...
    SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp]);
...

Calculating which tile should move

In order to play the puzzle, the device can be tilted in four directions:

up (the front camera tilts away from the user)
down (the microphone tilts towards the user)
left (right side of the device goes up)
right (left side of the device goes up)

Let's create an enum that will be used to describe the device's movement. It will also contain the value none, to describe the device in a state that we consider "flat" or "untilted".

We will condense Tilt into a single Movement value, since we assume that the phone can only be tilted in one direction at any single point in time. We simply consider the largest angle to be the dominant direction.

A device is tilted in a certain direction if the tilt is larger than (arbitrarily selected) 5 degrees. Any tilt smaller than that will be considered as laying flat. We'll make an extension function on Tilt that we'll use to get our current Movement direction.

// in lib/movement/bloc/movement.dart
import 'package:tilt/tilt.dart';

enum Movement { up, down, left, right, none }

extension T on Tilt {
  Movement get movementDirection {
    final absXDeg = xDegrees.abs();
    final absYDeg = yDegrees.abs();
    if (absXDeg < _triggerMovementDegrees && absYDeg < _triggerMovementDegrees) {
      return Movement.none;
    }
    if (absXDeg > absYDeg) {
      return xRadian.isNegative ? Movement.up : Movement.down;
    } else {
      return yRadian.isNegative ? Movement.left : Movement.right;
    }
  }
}

const _triggerMovementDegrees = 5;

You can verify that it works as intended by emitting event.movementDirection.toString() from your MovementCubit. You'll see the current Movement displayed bellow the puzzle board, and observe it changing as you rotate your device in different directions.

Getting the puzzle state

Your MovementCubit now has information about device's tilt, but in order to start using the tilt data to control the puzzle, it first must know the state of the board that is being controlled. This means that we need to find a way to get information about PuzzleState into MovementCubit.

To set this up in the simplest way possible, we will pass a reference to PuzzleBloc to the MovementCubit.

⚠️ Note: This is not the way you would normally go about setting up interaction between two Blocs. A proper solution would probably include creating a service that contains shared properties that both Blocs can communicate with. However, this would require refactoring PuzzleBloc's code, which would break our original goal of leaving as much of original classes intact as possible.

// in lib/puzzle/view/puzzle_page.dart
...
          child: MultiBlocProvider(
            providers: [
              BlocProvider(
                create: (context) => TimerBloc(
                  ticker: const Ticker(),
                ),
              ),
              BlocProvider(
                create: (context) => PuzzleBloc(4)
                  ..add(
                    PuzzleInitialized(
                      shufflePuzzle: shufflePuzzle,
                    ),
                  ),
              ),
              BlocProvider(
                create: (context) => MovementCubit(context.read<PuzzleBloc>()), // get a reference to `PuzzleBloc`
              ),
            ],
...

Which tile will move?

Now that you have access to PuzzleState, as well as information about which way the user is tilting their phone, you can determine which Tile (if any) is about to be moved. To accomplish this, we will create a few extensions on the original models.

I like putting these extensions in a separate part file, as a way of keeping the MovementCubit readable and easy to navigate, while still keeping the functionality private (i.e., only available to the Cubit that needs them). You can choose whichever location you are comfortable with.

// in lib/movement/bloc/puzzle_helper.dart
part of 'movement_cubit.dart';

extension on Puzzle {
  Tile? movableTile(Movement direction) {
    final candidatePosition = getWhitespaceTile().position(direction);
    try {
      return tiles.firstWhere(
        (tile) => tile.currentPosition == candidatePosition,
      );
    } catch (e) {
      return null;
    }
  }
}

extension on Tile {
  Position position(Movement direction) {
    switch (direction) {
      case Movement.none:
        return currentPosition;
      case Movement.up:
        return currentPosition.copyWith(y: currentPosition.y + 1);
      case Movement.down:
        return currentPosition.copyWith(y: currentPosition.y - 1);
      case Movement.left:
        return currentPosition.copyWith(x: currentPosition.x + 1);
      case Movement.right:
        return currentPosition.copyWith(x: currentPosition.x - 1);
    }
  }
}

extension on Position {
  Position copyWith({int? x, int? y}) => Position(x: x ?? this.x, y: y ?? this.y);
}

Starting at the bottom of the file, we're defining a simple copyWith method on Position, which makes the next chunk of code more readable.

In Tile.position, we are asking a Tile to give us a Position that describes which of its neighboring tiles would bump into it if the phone tilted in the given direction. Since the empty slot on our puzzle board is also a Tile, we can also think about this method as calculating which tile (it any) would slide into the empty space if the phone as tilted a certain direction.

For example, if the empty slot is at a position [1,1] (this would be the one in the top left corner) and the phone is tilting in direction of Movement.up, we'll get back Position(1,2) (which describes first tile of the second row).

Note that for tiles at the edges of the board the neighbor's position is sometimes impossible: we can't place a tile at Position(x: 0, y: 0). This, however, won't present a problem, because in the next step we will take into account only existing tiles.

Puzzle.movableTile is the brain of the operation. It first finds out which tile is "whitespace". Next, in whitespace.position(direction) it finds which Tile should theoretically slide into the empty slot, given the direction at which the phone is being tilted. And finally, it looks at the list of real available tiles, and returns the Tile whose Position matches the theoretical value. If there is no such tile, it means that the empty slot is at the edge of the board, and we simply return null.

This might be a lot to take in at once, so let's try visualizing what happens.

Visualize which tile is going to move

Let's first create a new class that will describe the complete state of our UI. We will call it MovementState, and make it MovementCubit's state.

We have (mostly arbitrarily) selected the value of 22.5° to be the angle that needs to be reached before we consider the phone tilted enough for a Tile to slip out of its current slot and slide into the empty slot.

With xTilt and yTilt we are describing the percentage of the required tilt reached: e.g., when a phone is tilted 15 degrees, it is two thirds (0.666...) of the way there, and the user needs to tilt it 7.5 degrees more in the same direction before the tile can fall into the empty slot.

// in lib/movement/bloc/movement_state.dart
import 'package:tilt/tilt.dart';
import 'package:very_good_slide_puzzle/models/tile.dart';

const tiltTriggerDegrees = 22.5;

class MovementState {
  MovementState._(
this.tilt,
 this.movementCandidate);

  factory MovementState.initial() => MovementState._(const Tilt(0, 0), null);

  final Tilt tilt;
  final Tile? movementCandidate;

  double get xTilt => tilt.xDegrees.abs() > tilt.yDegrees.abs() ? (tilt.xDegrees / tiltTriggerDegrees).clamp(-1, 1) : 0;
  double get yTilt => tilt.yDegrees.abs() > tilt.xDegrees.abs() ? (tilt.yDegrees / tiltTriggerDegrees).clamp(-1, 1) : 0;

  MovementState copyWith(Tilt? tilt, Tile? movementCandidate) => MovementState._(
        tilt ?? this.tilt,
        movementCandidate ?? this.movementCandidate,
      );
}

Our MovementCubit now has to determine which direction the phone is being tilted based on Tilt. Next, using that information, it will pick from Puzzle either one or zero Tiles that should be moved based on this direction. Finally, the cubit will emit this information as MovementState.

On the presentation side of things, we will override tileBuilder method of our MovementLayoutDelegate. For each tile we will check whether it is potentially able to slide into the empty slot, and if so, we will offset its position to indicate this.

// in lib/movement/bloc/movement_cubit.dart
...
class MovementCubit extends Cubit<MovementState> {
  MovementCubit(this._puzzleBloc) : super(MovementState.initial()) {
    SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp]);
    DeviceTilt().stream.listen((tilt) {
      final direction = tilt.movementDirection;
      final movementCandidate = _puzzleBloc.state.puzzle.movableTile(direction);
      emit(state.copyWith(tilt, movementCandidate));
    });
  }
  final PuzzleBloc _puzzleBloc;
}

// in lib/movement/presentation/movement_layout_delegate.dart
...
  @override
  Widget tileBuilder(Tile tile, PuzzleState state) {
    return LayoutBuilder(
      builder: (context, constraints) {
        final movementState = context.select((MovementCubit cubit) => cubit.state);
        final movementCandidate = movementState.movementCandidate;
        // if this tile is not elegible for movement, return the original view
        if (movementCandidate == null || movementCandidate.currentPosition != tile.currentPosition) {
          return super.tileBuilder(tile, state);
        }
        // Offset the position of the movement candidate tile in a given direction
        final halfTile = constraints.maxWidth / 2;
        return AnimatedContainer(
          duration: const Duration(milliseconds: 100),
          curve: Curves.slowMiddle,
          transform: Matrix4.translationValues(
            movementState.yTilt * halfTile,
            movementState.xTilt * halfTile,
            0,
          ),
          child: super.tileBuilder(tile, state),
        );
      },
    );
  }
...

As you might remember from earlier, tiltX and tiltY tell us how close the current tilt is to what we consider to be large enough angle to trigger the tile to change its position. In the code above, we're using this information to visually offset (translate) the tile that the user is manipulating.

If you run the app now, you will see that tilting the phone results in the corresponding tile slowly approaching the whitespace slot, but never quite snapping into it. This is because we are clamping xTilt and yTilt so that they don't register values higher than tiltTriggerDegrees, while also making sure that translation offset is never bigger than halfTile. This way we will never give the user an idea that the tile has slipped into a new position without it actually happening.

💡It's important to note here that as far as the PuzzleBloc is concerned, there is no change in PuzzleState and the way that the user interacts with the game. This means that all the PuzzleBloc's tests are still (correctly) testing the original functionality and passing, and that the other two modes of the game ("simple" and "dashatar") are completely unaffected by the functionality we added.

Finally, controlling the game

To make the tile snap into the empty slot, we simply need to add TileTapped event to PuzzleBloc once tiltTriggerDegrees is reached. This approach, however, is a bit too naive. Let's demonstrate why by actually implementing it.

// in lib/movement/bloc/movement_cubit.dart
 ...
      final direction = tilt.movementDirection;
      final movementCandidate = _puzzleBloc.state.puzzle.movableTile(direction);
      emit(state.copyWith(tilt, movementCandidate));
      if (state.xTilt.abs() == 1 || state.yTilt.abs() == 1) {
        _puzzleBloc.add(TileTapped(movementCandidate!));
      }
...

If you run the app now, you will see that, indeed, the tiles finally move in response to you moving your phone. However, they seem to move a bit too eagerly: once you tilt your phone in a certain direction, it triggers movement of all tiles that can move in that direction, resulting in entire rows or columns of tiles sliding into their new positions at once.

This is because once the tiltTriggerDegrees has been reached, the TileTapped event is fired for each sequential tile that fulfills the conditions, and this happens faster than a human user can readjust the angle to prevent the next tile from falling into the newly empty slot.

Making the game playable

One way to account for human imprecision in our puzzle is to make MovementState aware of user's previous move, so that MovementCubit can take it into account when determining whether to trigger TileTapped or not.

Let's add swapHistory to MovementState. It will be a property that keeps track of all the tile movements that happened in a game.

// in lib/movement/bloc/movement_state.dart
...
class MovementState {
  MovementState._(this.tilt, this.movementCandidate, this.swapHistory);

  factory MovementState.initial() => MovementState._(
        const Tilt(0, 0),
        null,
        const [Movement.none],
      );
  final List<Movement> swapHistory;
...
  MovementState swap(Movement direction) => MovementState._(
        tilt,
        null,
        [...swapHistory, direction],
      );

  MovementState copyWith(Tilt? tilt, Tile? movementCandidate) => MovementState._(
        tilt ?? this.tilt,
        movementCandidate ?? this.movementCandidate,
        swapHistory,
      );
}
...

We will then add a _shouldSwap method to the MovementCubit, and increase the tilt threshold for each subsequent movement in the same direction. From the user's perspective, this means that in order to move two tiles in the same row or column, they will have to tilt their phone a bit further to trigger the movement of the second tile.

// in lib/movement/bloc/movement_cubit.dart
...
    // inside `DeviceTilt` listener
    final s = state.copyWith(tilt, movementCandidate);
    if (_shouldSwap(s)) {
      _puzzleBloc.add(TileTapped(movementCandidate!));
      emit(s.swap(s.tilt.movementDirection));
    } else {
      emit(s);
    }
...
  // method of `MovementCubit`
  bool _shouldSwap(MovementState currentState) {
    if (currentState.movementCandidate == null) {
      return false;
    }

    final direction = currentState.tilt.movementDirection;
    final previousMovementsInTheSameDirection =
        currentState.swapHistory.reversed.takeWhile((value) => value == direction);

    var requiredTilt = tiltTriggerDegrees;
    for (var i = 0; i < previousMovementsInTheSameDirection.length; i++) {
      requiredTilt += requiredTilt / 3 * 2;
    }

    late final double currentTilt;

    switch (direction) {
      case Movement.up:
      case Movement.down:
        currentTilt = currentState.tilt.xDegrees.abs();
        break;
      case Movement.left:
      case Movement.right:
        currentTilt = currentState.tilt.yDegrees.abs();
        break;
      case Movement.none:
        return false;
    }

    return currentTilt > requiredTilt;
  }
...

_shouldSwap first checks whether there is a Tile that would move in a given direction, and immediately returns false if there is none. Next, it checks which direction is the phone currently being tilted, and how many consecutive swaps have already been made in this direction.

It then increases the requiredTilt by 2/3 for each tile already moved in the same row or column. It calculates the relevant value for currentTilt, which depends on the direction the phone is being tilted. And finally, it determines whether a swap should occur based on whether currentTilt is exceeding requiredTilt.

Running the app now, you will see that the puzzle behaves as expected: it is possible to move just one tile at the time, but doing so still requires dexterity and precision of movement that make playing the game a fun challenge.

Finishing touches

There is one more quick improvement we could add to make the app feel more polished. Since our game mode is all about the user physically moving their phone around, we can add subtle vibrations as additional hits, thus making the game feel more immersive and "physical". However, bear in mind that this feature is not available on most tablets, so only some of our users will be able to benefit from this.

Flutter provides easy access to HapticFeedback class. We can add a HapticFeedback.heavyImpact() vibration when a TileTapped is triggered, giving the user a satisfying tactile feedback that the tile has snapped into place.

For tiles that are about to be swapped, we can add a more subtle vibration, and thanks to how often our code is being updated with new values from sensors, HapticFeedback.lightImpact() will feel more like a continuous vibration than discrete impacts. We'll start with the haptic feedback if a tile is at 90% or more of the tilt required for it to be swapped.

// in lib/movement/bloc/movement_cubit.dart
...
	_puzzleBloc.add(TileTapped(movementCandidate!));
	HapticFeedback.heavyImpact();
...
 bool _shouldSwap(MovementState currentState) {
...
    if (currentTilt > requiredTilt * 0.9) {
      HapticFeedback.lightImpact();
    }
}

Extra credit assignments

The app now looks good and works as expected, but there are several things you can do to make it even better.

First, you could make tiltTriggerDegrees depend on puzzle board size. Current fixed value is acceptable for a 4x4 board setup, because the biggest tilt you need to make to move a tile is 62.5°. By enlarging the board to just 5x5, the required tilt for the last tile in a row or a column becomes 104°! At this point the game is not comfortable to play anymore.

Second, you could use the swapHistory property to quite easily add "Undo" functionality. All you'd need to do is figure out how to make movements opposite of what's in swapHistory. An even cooler use of swapHistory would be a "Replay" functionality, where you could replicate the user's entire game upon successfully solving the puzzle (maybe with some fancy animations?).

Thirdly, if you're planning to use swapHistory, you will also need to make sure that you clear the history if the "Shuffle" button is pressed. Right now you're saving history for all the games played until the app is killed, which is not a problem in itself (it could even turn into a feature), until you start using swapHistory for different purposes.

And finally, you can make tilting animations for each subsequent tile in a row or a column feel more natural by including the new, higher tilt requirements while calculating xTilt and yTilt. This could make swapping multiple tiles in a same row or column feel more natural.

Conclusion

You've reached the end of this tutorial!

If everything went right, you should now have a fun, playable puzzle app that is cool because of functionality that you added yourself! Congratulations! 🎉

Here are some key skills I hope you took from this tutorial:

examining architecture and extending it without awkwardly forcing new functionality into existing classes
approaching the problem in a step-by-step manner, and testing your solution early and often (even though in this case testing was just visual inspection)
some enjoyment from interacting with your phone not in the usual 'swipe-finger-on-touchscreen', but in a more direct, physical way

Next up, in the spirit of Flutter's cross-platform nature, I will look into how to extend the fun features we added here to desktop platforms. Desktop platforms generally lack both gyroscope and accelerometer sensors, and are not as easy to physically move around, so we will have to think of a different way of controlling the game.

Stay tuned!