• in

    Golang – Cross Platform Building

    In my work, I write code on a Mac and deploy to work environments like Windows Server or Centos Linux.

    So I needed to know how to build cross platform.  Running “go build” just built for my local machine.

    I came across a video dealing with the subject:

    In my case, I didn’t follow the entirety of his advice.  I had already installed XCode in the past, so I didn’t need to jump through those hoops.

    All I needed was to run these params on the build command line, which set the OS and Architecture for the target device:

    GOOS=linux GOARCH=amd64 go build portScanner.go

    The above example sets the build process to make a binary for the Linux OS using the amd64 bit chipset.

    I was able to do the same for Windows.  I’m not sure of all the values available.  I noticed I couldn’t specify intel64 or intel32 or x86 for Windows.  In my app, I was able to specify amd64 for Windows (even though it was an intel chipset) and the exe binary worked fine.

     

  • in

    Go – GoRoutines, Channels and Threads

    Each thing I learn in Go, I find myself trying to refer to it as I would in another language (thinking of structs as Classes, for example.)

    There are however some differences with the Go implementation of these concepts.  In this post I wanted to take some material I’ve been reading about and learning about in various courses and books.  Specifically, I’d like to refer to the concept of concurrency.

    GoRoutines

    A goroutine is a way of running functions concurrently.  In Grails there is a concept of using “promises” and tasks… and this is kinda similar.  But keep in mind that concurrent doesn’t necessarily mean parallel.

    In my Port Scanner, I was originally running each check on a port in a range sequentially.  It would take several min to iterate over a list of 8000 ports.  However, when I applied the concept of GoRoutines, I got this down to 20 seconds to run over 8,000 localhost ports.

    Setting Up GoRoutines

    To make use of GoRoutines we need to define a wait group.  The concept of a WaitGroup works like this:

    1. we define a wait group
    2. we define how many groups (concurrent functions) we’ll be doing
    3. we call the function using the “go” keyword (defining it as a Go Routine)
    4. Once the function process is done, we mark the group as done.

    In practicality it would look like this:

    var wg sync.WaitGroup
    wg.Add(4)		// 3s to run 1000 ports on 4 concurrent groups
    	go check_port(host, start_port, end_port_set1)
    	go check_port(host, (end_port_set1 + 1), end_port_set2)
    	go check_port(host, (end_port_set2 + 1), end_port_set3)
    	go check_port(host, (end_port_set3 + 1), end_port)
    	wg.Wait()
    
    func check_port(host string, start_port, end_port int) {
    
    	for i := start_port; i <= end_port; i++ {
    		//fmt.Println('\n')
    		qualified_host := fmt.Sprintf("%s%s%d", host, ":", i)
    		conn, err := net.DialTimeout("tcp", qualified_host, 10*time.Millisecond)  // Got the timeout code from: https://stackoverflow.com/questions/37294052/golang-why-net-dialtimeout-get-timeout-half-of-the-time
    		if err != nil {
    			continue
    		}
    		fmt.Fprintf(conn, "GET / HTTP/1.0\r\n\r\n1\n22\n\n\n\n")
    		conn.SetReadDeadline(time.Now().Add(10*time.Millisecond))
    
    		// swapping bufio reads to reading buffers as bytes gets huge performance increase: 8000 ports in 20s (vs 1min 10s using bufio reads)
    		buff := make([]byte, 1024)
    		n, _ := conn.Read(buff)
    		fmt.Printf("Port: %d%s\n",i, buff[:n])
    	}
    	wg.Done()
    }

    The first line defines the variable wg to hold the value for the WaitGroup method of the sync package (which will need to be imported.)

    wg then makes use of the built in method Add() and in this case I put in the value of 4 wg.Add(4) which will run 4 concurrent runs.

    Each concurrent call is defined as:

    go [function name](params)

    In this case:

    go check_port(host, start_port, end_port_set1)…

    At the bottom of this list of calls there’s a wait invoked: wg.Wait()

    Lastly, the function itself has a wg.Done() at the end of the logic.

    Channels in Go

    Channels are a complex subject for me.  At first, I thought it was the magic that made concurrency happen in Go. Instead, I discovered that GoRoutines are how concurrency is applied in a go app.  Channels offer the ability for GoRoutines to share data back and forth.

    I initially thought of channels as a “Thread,” as in other languages.

    However, in reading this programmer’s blog post, I discovered some differences to threads: http://tleyden.github.io/blog/2014/10/30/goroutines-vs-threads/

    You can see their full bulleted list of differences there.  I’ll just mention a couple highlights:

    1. GoRoutines can be run at a factor more than Threads on a system.
    2. GoRoutines have a mechanism to share and talk between GoRoutines (channels.)

    Channels are mechanisms to share data between GoRoutines.

    More info on channels:

  • in

    Hot Popular

    Go and OOP

    Notes from the course Learn How to Code: Google’s Go Programming Language

    Go and Types

    Like all languages values are of certain types.  Types can be int, string, boolean, etc.  Within Go, you can also define your own types.

    type hacker bool
    
    func main(){
    	var eliot hacker = true
    	fmt.Println(eliot)
    	fmt.Printf("%T ", eliot)
    }

    In the above example, eliot is set to a new type called “hacker.”  As we know in Mr. Robot, Eliot is a hacker, so this type has the underlying type of boolean.

    In the main function, eliot is set to true.  Verifying this and its type is done with the Print statements, which output:

    true
    main.hacker 

    OOP and Go

    In a post from 2013, Bill Kennedy writes:

    I consider Go to be a light object oriented programming language. Yes it does have encapsulation and type member functions but it lacks inheritance and therefore traditional polymorphism. For me, inheritance is useless unless you want to implement polymorphism. With the way interfaces are implemented in Go, there is no need for inheritance. Go took the best parts of OOP, left out the rest and gave us a better way to write polymorphic code.  –From Goinggo.net

    If you search long enough, you’ll soon discover that this topic is covered (and debated) quite a bit.  Yes, Go is Object Oriented, but it lacks the traditional elements we associate with Object Oriented Languages (like classes, inheritance, etc.)

    In Todd McLeod’s course on Go [link] he states that Go is OOP with these elements you would find in other OOP languages:

    • Encapsulation
    • Reusability through inheritance or embedded types
    • Polymorphism through interfaces
    • Overriding through promotion

    The above seems a tad contrary to some of the other links and discussion that I’ve come across.  However, Todd makes a good argument for these points.

    Regarding Encapsulation, Todd states that by creating our own types and that type has fields that hold state.  Those types can be associated to behaviors (like methods.)

    For Inheritance, Todd adequately shows in the course that a type can be injected anonymously into code.  When done, all the fields that were part of the struct are carried (promoted) with the injection.

    Example:

    type Car struct {
    	make string
    	model string
    	year int
    }
    
    type Demolition struct {
    	Car
    }

    As for Polymorphism, Todd McLeod explains that interfaces fit the bill.

    Finally, in regards to Overriding methods/functions, Todd McLeod shows that through “promotion” a function can be overridden.  This is a complex subject for me.  However, in watching and rewatching the lecture several times I have an idea of what he’s getting at. Below is a code example I wrote, that follows his own:

    
    type Car struct {
    	Make string
    	Model string
    	Year int
    }
    
    type Demolition struct {
    	Car
    	Make string
    	Convertable bool
    }
    
    func main(){
    	mustang := Demolition{
    		Car: Car{
    			Make: "Ford",
    			Model: "Mustang",
    			Year: 1973,
    		},
    		Make: "FORD MOTORS",
    		Convertable: true,
    	}
    
    	fmt.Println(mustang.Make, mustang.Model,  mustang.Year, mustang.Car.Make)
    }

    In the above code the output will be:

    FORD MOTORS Mustang 1973 Ford

    mustang.Make prints out “FORD MOTORS” and mustang.Car.Make will return the value “Ford.”  What occurred here is that the Car struct was promoted into the Demolition struct and and when we set a value to Demotion, we created a set of values for the mustang car.  However, another definition of “Make:” was created which shows overriding the initial value.

    Structs in Go

    The reason why I started with the concept of Types is that the instructor (Todd McLeod) brought up the idea first before going into Structs.  Here’s why – When making Structs in Go, you define a type of struct.

    A struct is a data set that resembles (at a glance) something akin to a class in other languages.  Example:

    type car struct {
    	make string
    	model string
    	year int
    }
    
    func main(){
    	mustang := car{"Ford", "Mustang", 1973}
    	fmt.Println(mustang.make, mustang.model,  mustang.year)
    }

    The value mustang is of type car.  It is reminiscent of a class being instantiated as objects.  I’m not sure if that’s what’s actually happening… and my understanding of the deep waters of OOP is quite limited.

     

  • in

    Trending Hot Popular

    Port Scanner written in Go

    Note: Only utilize a port scanner on sites you have permission to test. Also, note the legalities of port scanners in your territories.

    Update 8/13/17: I made a few modifications to the code. Namely:

    1. some software answers on a port but doesn’t respond and isn’t closed causing a hang. I put a timeout on the read action
    2. I used 4 go routines to add concurrency.
    3. I removed the bufio ReadString methods in place of raw bytes read in the buffer using this users methodology: https://gist.github.com/kenshinx/5796276  It improved the performance from 8000 ports scaned in over a min, to under 20 seconds!

    Go Project

    As I’m learning the Go language, I thought I’d work on a small project of making a utility I wrote in Python, in the Go language.  In this case, I chose a port scanner I wrote in Python.  The scanner tries to interrogate ports and return any banner that would indicate what’s installed there.

    The original code I wrote in Python utilized the socket and sys libraries.  I wrote it while taking an InfoSec course over at Offensive Security.  Anyway… years later, I’m thinking of a project to try and do in Go (just go kick the tires and get used to the code) and I figure I might as well port that project into Go.

    Go Port Scanner

    The code is simple it basically kicks off a shell prompt asking for a host, then a starting port and an ending port.  It then iterates over the port range, trying each port in turn.

    Here’s the code (link):

    package main
    
    import (
    	"fmt"
    	"net"
    	"time"
    	"sync"
    )
    
    var host string
    var start_port int
    var end_port int
    var wg sync.WaitGroup
    
    func main() {
    	user_input()
    }
    
    func check_port(host string, start_port, end_port int) {
    
    	for i := start_port; i <= end_port; i++ {
    		//fmt.Println('\n')
    		qualified_host := fmt.Sprintf("%s%s%d", host, ":", i)
    		conn, err := net.DialTimeout("tcp", qualified_host, 10*time.Millisecond)  // Got the timeout code from: https://stackoverflow.com/questions/37294052/golang-why-net-dialtimeout-get-timeout-half-of-the-time
    		if err != nil {
    			continue
    		}
    		fmt.Fprintf(conn, "GET / HTTP/1.0\r\n\r\n1\n22\n\n\n\n")
    		conn.SetReadDeadline(time.Now().Add(10*time.Millisecond))
    
    		// swapping bufio reads to reading buffers as bytes gets huge performance increase: 8000 ports in 20s (vs 1min 10s using bufio reads)
    		buff := make([]byte, 1024)
    		n, _ := conn.Read(buff)
    		fmt.Printf("Port: %d%s\n",i, buff[:n])
    	}
    	wg.Done()
    }
    
    func user_input() {
    	fmt.Println("Host> ")
    	fmt.Scan(&host)
    	fmt.Println("Starting Port (i.e. 80)> ")
    	fmt.Scan(&start_port)
    	fmt.Println("End Port (i.e. 8080)> ")
    	fmt.Scan(&end_port)
    	fmt.Println("Running scan... ")
    
    	//check_port_set1(host, start_port, end_port) // 15s to run 1000 ports sequentially
    
    	port_range := end_port - start_port
    	end_port_set1 := (port_range / 4) + start_port
    	end_port_set2 := (port_range / 4) + end_port_set1
    	end_port_set3 := (port_range / 4) + end_port_set2
    
    	wg.Add(4)		// 3s to run 1000 ports on 4 concurrent groups
    	go check_port(host, start_port, end_port_set1)
    	go check_port(host, (end_port_set1 + 1), end_port_set2)
    	go check_port(host, (end_port_set2 + 1), end_port_set3)
    	go check_port(host, (end_port_set3 + 1), end_port)
    	wg.Wait()
    
    }
  • in

    Hot Popular

    Go Basics – Defer

    In the Go language there is a concept of deferring an action till later.

    As far as I can tell, the usefulness of this is to allow for the code to be more readable.   In Todd McLeod’s course on Go, he offers an example of this used in an application he wrote. In the app a file is opened, some actions are done to the opened file (such as writing content to it) and then the file is closed.  Instead of having the close at the bottom of the function, he has the close call under the open call.

    Example:

    ipfile, err := os.Open(“ipaddresses.txt”)

    defer ipfile.Close()

    In the above example, the close is near the call to open the file.  The work done on the file is below the defer action.  Before the function closes, it runs the defer actions

    As far as I know… this helps in readability (i.e. we put the open and close actions together), but has no real functionality impact as this could easily have been done using the close call at the end of the function.

  • in

    Trending Hot Popular

    Go – Variadic Functions

    I hadn’t come across this concept in previous languages I worked in.  In Go, however, this concept came up in an online course I was taking on Udemy.com (from Todd McLeod.)  He raised the concept of a Variadic Function.

    Variadic Functions

    These are functions which take an unspecified amount of parameters.  In a regular function you would lock down what parameters are allowed (how many and of what types.)  In a Variadic Function, you use “…” notation to allow for an undefined amount values to get passed.

    Languages Using Variadic Functions

    Go is not the only language to use Variadic Functions.  Other languages that make use of this (even using the same triple dot notation) are:

    • C++
    • Java
    • JavaScript

    Python and Ruby also make use of this concept, although with a different notation (the asterisk.)

    Examples of Variadic Functions in Go

    Here’s some code showing a variadic function in Golang:

    package main
    import "fmt"
    
    func main(){
    	variadicsf("Brian", "Austin", "Lisa", "Travis", "Dan")
    }
    
    func variadicsf(names ...string){
    	fmt.Println(names)
    
    	for _, name := range names {
    		fmt.Println(name)
    	}
    }
    
    // The Variadic function allows for multiple params to be passed in.
    // Notation of this is in the form of ...
    // These params are put into an array.
    // Also note the for loop making use of a blank identifier (the underscore)
    // The blank identifier allows me to ignore the key of the array and only concern myself with the value.

    I the above example, the code will output the collection/array (fmt.Println(names)) as well loop through the array and output each name passed in.  Here’s the outuput:

    [Brian Austin Lisa Travis Dan]
    Brian
    Austin
    Lisa
    Travis
    Dan

    Other Variadic Forms

    It is possible to have a variable function and you want to pass into it data from a slice.  The data could be like this (taken from Todd McLeod’s course): data := []float64{43, 24,10,24,33,54}

    That data can not simply be passed into a variadic function.  Instead, the call to the function needs to be modified like so:

    multip(data…)

    func multip(num …float64){ }

    What is occurring in this situation is that the call to the method is using a variadic parameter (triple dots appended to the param.)  This sends each element to the function, one at a time.  So first 43 is sent into the multip function, then 24, then 10 and so on.

     

    For more info on these topics, check out Todd McLeod’s course: Learn to Code using Go

  • in

    Hot Popular

    Go – Reversing a String

    As a little example of basic Golang, I wrote a little app that takes user input and reverses the word as output:

    package main
    
    import "fmt"
    
    var userInput string
    
    func main() {
       reverseWord()
    }
    
    func reverseWord() {
    
       fmt.Println("Input a word> ")
       fmt.Scan(&userInput)
       var word_length int = len(userInput)
       for i := word_length; i > 0; i-- {
          fmt.Print(userInput[(i - 1):i])
       }
    }
    
    		
  • in

    Heart Rate Tracking

    The Goal

    I’ve always wanted a monitor of bio data… specifically data related to heart rate and GSR (Galvanic Skin Response) and if possible EEG data as well.

    I’d love to have some giant monitor set up with streaming data on a variety of subjects.  For now… baby steps.

    Tracking Data with Fitbit

    Fitbit is a nice little wearable monitoring system that helps people keep track of their fitness goals.  For me, I just wanted it so that I could monitor my heart rate.  I thought about several fitness trackers, most many of them do not capture your heart rate ALL DAY LONG.  Nor do they do so on each second of the day.  There are some however that do… Fitbit does, as does Jawbone.  I looked into Apple watch, but everything I read was that it was going for different goals and didn’t track each heartbeat throughout the day.

    For me, it was a toss up between Fitbit or Jawbone Up3… Looking back, I may have been better off picking the Jawbone, as it has a GSR sensor as well.

    The Problem I’m trying to Solve

    While the Fitbit tracker is pretty nifty it (nor its app) didn’t meet the following criteria:

    • Intra-Day data – By default, the app shows the past 24hr of data. I wanted it down to the min.
    • Push this data public, so anyone with a browser can see my current heart rate.

    That 24hr window was too annoying.  If the data was captured intra-day (like every min, every hour, etc.) I could get a higher resolution of my heart rate and could correlate changes and fluctuations to life events more easily.

    Thankfully the Fitbit team allow a user to access their intra-day data that is collected by the device.  They capture a heart rate every 1 second.  This data is passed to the phone (via the Fitbit App) and pushed to the Fitbit servers.

    Fitbit API Limitations

    The Fitbit API is limited to 150 calls per hour.

    I am only allowed to display my personal data (i.e. I can’t pass other user’s through their login to access their intra-day data.)

    Angular2 & Fitbit

    While there are many different Node and Angular libraries out there – I struck out getting any / all of them to work.  Instead, I opted to do this all by hand.  I would create a method to capture an authorization token, then pass it along with the header to get my intra-day data.  After collecting the data, I choose to store it in a Firebase database.  The database is being subscribed to via web sockets, so any data updates will kick off a dynamic update on the page.

    As I was studying Angular2 at the time, I choose to do the work in Angular2.  The actual call to grab the user data should be done as a backend process, but for now, it’s front end.  But any server could generate this call and store the result in Firebase.

    Angular2 App Setup

    My app was a CLI type, so I used the command line: ng g component home

    This created my “home” component (a dashboard of sorts.) This is going to be the landing page I’ll set with routes later on.

    Firebase Setup

    Why Firebase?  Well for my needs, it’s free.  It’s a Google product and they throw in a free SSL cert.  Their Databae is realtime and pretty amazing.  Everything is stored in the Non-SQL db as objects, so pulling it out and iterating / parsing the data is pretty straight forward.  It also works well with Angular and JS frameworks.

    For Angular2, run: ng install angularfire2 from the project root and that will set it up for your app.

    I won’t cover much on Firebase, but you can check it out at firebase.google.com and create an account.

    Once you have the account, you go to your console and click on Database.

    In the DB screen, you’ll see your db url (https://yoursite-firebase.blahblah) and you can simply append a model, like so:

    /posts to the end of your url and hit enter… it creates the table for you!

    If you go back to your Firebase dashboard for this project, you should see this:

    screen-shot-2016-12-27-at-3-32-39-pm

     

    Clicking “Add Firebase to your web app” will give you a popup with the connection data you’ll need to input in your apps.  You’ll need this to connect to firebase to read or save data.

    Next, back in the angular app, I created added to the app.module.ts file a config to connect to Firebase… from the “Add Firebase to your web app” popup:

    export const firebaseConfig = {
      apiKey: "some key goes here",
      authDomain: "my domain info goes here",
      databaseURL: "url to my database here",
      storageBucket: "same drill as above",
      messagingSenderId: "my firebase sender id here"
    }
    
    

    While in app.module.ts lets make a reference to a file not yet created:

    import {FirebaseService} from "./services/firebase.service";

    Firebase Service

    I created a services folder under the /src/app folder for the project and in there I created a firebase.service file.  This is where the firebase calls are going to get called from.

    I use the following imports:

    import {Injectable} from '@angular/core';
    import {AngularFire, FirebaseListObservable} from 'angularfire2';
    import 'rxjs/add/operator/map';
    import {AccessToken} from "../AccessToken";
    import {HeartRate} from "../HeartRate1Day";

    AccessToken and HeartRate are Models that for the data pulled and saved from/into Firebase.  You can create them if you like.  Further on, I’ll show how I used them.

    Adding a Fitbit Callback

    When you create your app on the Fitbit dev side, you’ll be asked to provide a callback… for now you can set it to your localhost/callback

    Back in the app, from the command line create a callback component:

    ng g component callback

    I use these imports:

    import { Component, OnInit } from '@angular/core';
    import {ActivatedRoute, Router} from "@angular/router";
    import { URLSearchParams, } from '@angular/http';
    import { FirebaseService } from '../services/firebase.service';

    Above the constructor I have defined a string of access_token:

    access_token: string;

    The constructor looks like this:

    constructor(public routes:Router, private firebaseService: FirebaseService) {
    
      routes.events.subscribe(s => {
    
          let params = new URLSearchParams(s.url.split('#')[1]);
          let access_token = params.get('access_token');
          var created_at = new Date().toString();
          var postContent = {
          access_token: access_token,
          created_at: created_at
        };
          console.log(postContent)
          this.firebaseService.addToken('put the db key here', postContent);
      });
    }

    In bold above, I have “put the db key here” this database element that we reference for the token.  You can create this in Firebase for your project.  I didn’t get into that granular level with Firebase, but it’s pretty easy, you can do it in the UI, with the + sign on a db table like [your firebase db]/token, or you could first post to firebase and it will generate a random string for the database element.

    Adding A Fitbit Token

    In the firebase.service file, modify the constructor to:

    constructor(private af: AngularFire) {
    
    }

    Above the constructor let’s create our acess_token variable:

    access_token: FirebaseListObservable<AccessToken[]>;

    You might be able to get away with <any[]>, but in my case I defined a Model called Accesstoken[].  This model exists in the /src/app folder and it’s contents are:

    AccessToken.ts:

    export interface AccessToken{
      $key?: string;
      token?: string;
      created_at?: string;
    }

    That “addToken” method isn’t yet created, so let’s make it.  Back in the firebase service, let’s create a method addToken, just below the

    addToken(key, newToken) {
      this.access_token = this.af.database.list('/token').map( (arr) => {return arr;}) as FirebaseListObservable<AccessToken[]>;
    
      return this.access_token.update(key, newToken);
    }

    The above is setting the access token from Fitbit as a FirebaseListObservable with our model AccessToken.  This value is then returned as an update to an existing reference (what’s passed in as the key.)

    Routes

    Routes are endpoint declarations that allow visitors to hit a specific endpoint and it pulls a corresponding Component.

    We need a route to /callback, for our Fitbit configuration to work.

    Back in angular.module.ts add this import:

    import {RouterModule, Routes} from '@angular/router';
    Then add this below the imports:
    
    const appRoutes: Routes = [
      {path: '', component: HomeComponent},
      {path: 'callback', component: CallbackComponent},
    ];
    
    

    The first route there is taking the empty string (just hitting the domain) and sending it to the home component we created.  Never use ‘/’ in Angular2 as a route path… it will barf pretty bad.

    The second route there is defining our callback.  When our Fitbit app makes a call to get a token, it redirects to this endpoint /callback.  We have set ‘callback’ (our endpoint) to go to the Callback component.

    In the @NgModule in angular.module.ts, add the following in the imports bit… somewhere in there, append:

    , RouterModule.forRoot(appRoutes)

    Finally, in app.component.html add this tag:

    <router-outlet></router-outlet>

    With this in place, here’s how the flow works:

    When index.html is called, it pulls the app-route tag, this calls the app.component.html – it’s tag to router-outlet will pull the router references we created in the module…

    Testing The Token Saving

    Back in the dev Fitbit site, go to your app (Manage Apps) and edit it.  There you can test it using the link at the bottom (OAuth 2.0 tutorial page).  This loads a test form.  Fill it out with your app’s details… including the callback of localhost:4200/callback (or whatever port you set up) – make sure that your callback here matches the callback in the app details!!

    Also, only the Fitbit app type of PERSONAL will allow you to get your intra-day data.

    Start up your local Angular2 app (if it isn’t running already.)

    When you test this Fitbit app in your OAuth tutorial page, it should pull up your localhost app on port 4200 with the /callback endpoint… this in turn should kick off your constructor on the callback.component.ts file.  Which has a method here to send the token key and value to: this.firebaseService.addToken method.

    This, in turn adds (or updates) our token.

    Check Firebase via browser …. go to your Firebase console, Project, Database and see if your table has a new entry (or an updated entry.) It should have the contents of the token in there.

    Getting Fitbit Heart Rate Data

    Ok, the token was the first part of our Ultimate quest: getting intra-day heart rate data.

    Back in the home.component.ts file, let’s add this method below our constructor:

    getMinHR(){
    
      var whatTime = new Date();
      var hour = whatTime.getHours();
      var minute = whatTime.getMinutes();
    
      this.firebaseService.getToken().subscribe(validtoken => {
        this.validtoken = validtoken;
        var headers = new Headers({'Authorization': 'Bearer ' + validtoken['access_token']});
        var formatedApiCall = 'https://api.fitbit.com/1/user/-/activities/heart/date/today/1d/1sec/time/'+(hour - 1)+':'+(minute - 1)+'/'+hour+':'+minute+'.json'
        this.http.get(formatedApiCall, {headers:headers})
          .map(res => console.log(this.firebaseService.saveMinHR(res.json())))
          .subscribe();
      });
    }

    The way the Fitbit api works for intraday, is that you pass in a time stamp into the Rest endpoint.  Read up on their intraday endpoint.  Since I wan to grab data from 1 hour ago, I take the current Date, pull the hours on it with the “getHours()” method, as well as the getMinutes() method.  I subtract the hour by 1 and pass that value into the rest call.  Along with the current time.  this gives me my 1 hour range in the Rest end point.

    Note the headers I’m using.  I’m getting a token from getToken() (a method we haven’t made yet) and subscribing to it.  I take the token as “validtoken” and I pass that into the call with the headers… To do this you’ll also need this import:

    import {Http, Headers} from "@angular/http";

    This lets you use http and pass header info into the http requests.  We want our header to have Authorization with a value of “Bearer and our valid toke”.

    Getting the Token

    I used a getToken() call so let’s go make that method.  In the firebase service, I made a method called getToken():

    getToken(){
      const token = this.af.database.object('/token/PUT YOUR DB ELEMENT HERE THAT YOUR TOKEN OBJECT SAVED TO');
      return token;
    }

    replace my text in the databse.object with whatever path is your token in the Firebase db.

    Getting The Heartrate Data

    This is then returned back to the getMinHR method.  Once we have the token it is sent along with the REST request and that allows us to pull our intra-day data as a JSON object.  This is sent to another Firebase Service method: saveMinHR().

    Saving (Updating) the Fitbit Intraday Heartrate Data

    Back in the Firebase service file, I have another method to save the heartrate data we just got.  The method simply does:

    saveHR(data){
      this.heart_rate = this.af.database.list('/hr').map( (arr) => {return arr}) as FirebaseListObservable<HeartRate[]>;
      return this.heart_rate.update('YOUR KEY THAT YOU'LL UPDATE',data);
    }

    Same as the token, my goal here isn’t to have a bunch of data entries for Heartrate.  Instead, I have a table /hr and in it an element that will be updated (a key, if you will.)  That can be created in the Firebase UI.  The update command will look up the key you have created in the /hr data table, and will pass in the data (your heart rate data from Fitbit.)

    I’m also using a model of HeartRate[] as the typing on the FirebaseListObservable.

    My HeartRate model is in the /src/app location and is HeartRate.ts (this file also needs to be imported into the firebase service file.):

    export interface HeartRate{
      $key?: string;
      data?: string;
    }

    Right now, if all went well, you are getting a token and using the token in a REST call to Fitbit, to get your heart rate data for the last hour (make sure you have sync’d your Fitbit to their mobile app, so that your Fitbit account has data for the past hour.)

    Charting the Heartrate Data

    Honestly I felt this was the hardest part.  I tried multiple chartin solutions, but ended up with Highcharts.js.  You’ll need to install Highcharts: https://www.npmjs.com/package/angular2-highcharts

    Highcharts had a simple line graph example that worked well for me.  It takes an array of data and plots the points.

    The data I was getting from Fitbit was being stored into Firebase as an object like so:

    /hr

    + [my key]

    +activities-heart

    +dataset

    +0

    ->  time: “16:36:13”
    -> value: 83

    screen-shot-2016-12-27-at-6-04-29-pm

    The time and value are what I want to plot here…

    After a lot of frustration, I got this to work like so:

    Heart Rate Componet

    Firt I created a new component, ng g component heartchart

    within heartchart.component.ts it looks like:

    import { Component, OnInit } from '@angular/core';
    import { FirebaseService } from '../services/firebase.service';
    @Component({
      selector: 'simple-chart-example',
      template: `<div *ngIf="options">
                    <chart [options]="options"></chart>
                </div>
        `
    })
    export class HeartChartComponent implements OnInit {
      hr: Object;
    
      constructor(private firebaseService: FirebaseService) {
    
        this.firebaseService.getMinHR().subscribe(hr =>{
          this.hr = hr;
          var timeValues = [];
          hr.forEach((k: any)=> {
            timeValues.push([k.time, k.value])
          });
    
          this.chartOptions(timeValues)
        });
      }
      options: Object;
    
      ngOnInit() {
      }
    
      chartOptions(data){
        this.options = {
          title : { text : 'Heart Rate Chart (Past Hour)' },
          xAxis: {
            title: {text: 'Time (Hover over Line for Timestamp)'},
            labels: {enabled:false}
          },
          yAxis: {title: {text: 'Heart Rate'}},
          series: [{
            name: 'Brians Heart Rate',
            data: data
          }]
        };
      }
    }

    Just looking at this makes me feel nausiated.  It took a lot of time to get data in the way that HighCharts wanted it…  Before getting into this bit… we need to create a new method on the firebase.service.ts.

    Not to confuse you, but I have two methods called getMinHR().  One is on the home.component.ts file and the other is in the firebase.service.ts file.

    In your Firebase Service, create another method for getMinHR() – this one will simply read data from Firebase:

    getMinHR(){
      this.current_hour_hr = this.af.database.list('/hr/YOUR DATA ELEMENT IN FIREBASE THAT YOU SAVE TO/activities-heart-intraday/dataset').map( (arr) => {return arr}) as FirebaseListObservable<HeartRate1Day[]>;
      return this.current_hour_hr;
    }

    This method, unlike the other, is getting the Heart Rate data from Firebase (what was saved.)  The other getMinHR() method was getting the data from Fitbit and saving it into Firebase.

    Question: why not just subscribe to Fitbit directly and not rely on saving the data to Firebase at all?

    Answer: I couldn’t get it working. Lame, but true.  I also was scared at the methodology that Angular2 might use to detect changes at the endpoint. I have limited usage of their endpoint (150 calls per hour.)  I opted to control the calls by setting a specific timer on pulling my data and saving it to Firebase.

    This method is calling the endpoint in Firebase (/hr/My KEY/activities-heart-intraday/dataset, recall from the break out, this will return data at this point:

    screen-shot-2016-12-27-at-6-04-29-pm

    Back in the HomeComponent.html I added this tag:

    <simple-chart-example></simple-chart-example>

    That’s the selector from my heartchart.component.ts file.

    Now when I go to the HomeComponent, it will load the chart (hopefully) with data returned.

    This works for me… it took me a couple weeks to get here, working on this nights and weekends.  A lot of trial and error (especially with graphing) went into this… I also utilized a lot of Stack Overflow and tutorials to get this fully working.

    End Result

    After all this, if you can get it working you’ll have a hearrate chart like this:

    screen-shot-2016-12-27-at-6-26-56-pm

    What’s great about smaller time sets is you have much greater resoution of your heartrate.  Instead of the default 24HR nonsense of Fitbit, you can zoom into 1min, 5min, or in this case 1hour incriments and see your heart reacting to your environment.

    In the image below I had a spike due to the stress of getting to work and finding that my parking garage was locked up, so it was very difficult to get in. This caused a spike in my heart rate:

    parking-heart-spike

    You can also see the chart in realtime at:

    https://hr.sdet.us

     

    Problems with Syncing & the Fitbit mobile App

    The biggest problem with this is the Fitbit sync.  For this chain of events to work, the wearable device needs to talk to the mobile app on my phone. It syncs my data and pushes it to the Fitbit server and I connect to that server with my token to retrieve it.

    Unfortunately, the Fitbit app stops syncing.  I’m not sure why.  It just stops.  When it stops I have to manually make it sync (pressing a manual sync button in the app itself) or relaunching the Fitbit app.  This is of course a nuisance.  If I neglect to manually sync and it stops auto syncing, the data will be very stale!

     

     

  • in

    Hot Popular

    Form Fuzzing with Python

    Another module in the Python standard library, is that of Mechanize.  Mechanize is a Python headless browser.  With Mechanize you can interact with web applications.  This could allow for headless verification of functionality (QA) or could be used in Penetration Testing.

    Proof of Concept with Python Mechanize

    In the following POC, I wrote a little script that interacted with some test web apps I created locally for testing.  This could be modified for a variety of different purposes, but here’s one version:

    import mechanize
     
    ''' Set up of Mechanize to support / fake a real browser '''
     
    mech = mechanize.Browser(factory=mechanize.RobustFactory())
    mech.set_handle_robots(False)
    ua = 'Mozilla/5.0 (X11; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0 (compatible;)'
    mech.addheaders = [('User-Agent', ua), ('Accept', '*/*')]
     
    # SQL Injection and Fuzz Values:
    badblood = ["'; or 1=1 a@gmail.com", "%5C%5C bill@gmail.com", "//jack@gmail.com","%20='"]
     
    def formScrape(url):
        # Initial form finding:
        mech.open(url)
        for form in mech.forms():
            print form
     
    def attackf(url, injection=badblood):
     
        for evil in badblood:
            print "Attempting attack with: " + evil
            try:
                mech.open(url)
                mech.select_form(nr=0)
                mech.form['j_username'] = evil
                mech.form['j_password'] = evil
                mech.submit()
                for line in mech.response():
                    print line
            except:
                pass
     
    attackf('https://127.0.0.1/login')

    The Methods Backdown

    There are two methods: formScrape() and attackf()

    formScrape takes a url as a parameter and uses Mechanize to search it for forms.  If it finds one or more forms it simply prints them out.

    attackf() also takes a required parameter of a URL, and also an optional parameter of a list.  I’m supplying a small list by default, but this could be modified to load a csv file or a dataset of fuzzer values.

    In the attackf method I iterate over each value in the bad characters/fuzz list and attempt to pass in those values into the forms’ fields.  In this case I have an example form with a username/password for a typical Java App.

    Finally I print out the results returned from the form post.

    Fuzzing Forms for Data Caps

    A typical fuzzer attempts to send data of some size, increase it and try again – repeating the process over and over until a threshold is reached or the application crashes.

    The same can be attempted on a web form as well,  Below is an example web form fuzzer using Mechanize:

    def formFuzzer(url, size=50):
        buffer=["\x00"]
        counter = 100
     
        while len(buffer ) &lt;= size:
            buffer.append("\x00"*counter)
            counter=counter+200
     
        for evil in buffer:
            print "Attempting attack with: " + evil
            try:
                mech.open(url)
                mech.select_form(nr=0)
                mech.form['search'] = evil
                mech.submit()
                for line in mech.response():
                    print line
            except:
                pass
     
     
    formFuzzer("https://127.0.0.1/someform",10)

    In this example, a buffer of null bytes is created based upon a size (option parameter) that is supplied.  This creates an array of various sizes of nullbytes and then below that, this array is iterated over to send those different sizes into the form.

    Instead of null bytes, it could be anything, such as a single character.

    Making it Useful

    To make these tests useful, tail the log files on the server you’re testing.  Verify that the conditions are handled correctly.  In some cases web applications can correctly handle the front end, but the back end might introduce an infinite loop. Watch for these in the error logs… as even if the XSS or SQL Injection code blocks bad POST’s into the database, you might trigger an inadvertent infinite loop in the error logs – which could bring down a server once the disk space is fully consumed.

    These simple PoC’s can be modified for testing web applications with Python – by observing the resulting behavior on your web application you can better spot and prevent future problems.For

  • in

    Go – Unused Code and the Blank Identifer

    Unused Code in Go (Golang)

    In the Go language (Golang) there is a concept that threw me for at first.  In other languages, you can have unused code sitting around.  By unused I don’t mean comments, instead I mean code that isn’t actively being consumed by the application.  For example, if I declared two variables (a and b) but only initialized variable a, even though the variable assignment for b is valid – it will not compile, as variable b is not getting consumed.

    This language requirement keeps the code useful.

    Blank Identifiers in Go (Golang)

    The underscore is the blank identifier.  It is used to stand in place of a variable assignment. For example, if you had a call to http.Get you’ll get back the response or an error.  Instead of writing the catch code to handle an error, you could call:

    resp, _ : = http.Get(….)

     

Load More
Congratulations. You've reached the end of the internet.