Uncategorized

Deploying Stuff from Win Machine To Linux via PowerShell

Introduction

When I was copying files or binaries from my Win machine into Raspberry or Azure VM, I was using WinSCP every time. It is ok when I have to do it once after a build. But when I have to copy it ten times in one hour,  because I am fixing a problem, it is annoying.

So I decided to automate it. It is possible to use WinSCP from command line. But there is a better option – PowerShell Module POSH-SSH, which contains CmdLets for conneting via SSH, SCP or SFTP.

 

Example – A Script to Copy Files on Linux Machine and Switch with Previous Version

# install latest version of the module
iex (New-Object Net.WebClient).DownloadString("https://gist.github.com/darkoperator/6152630/raw/c67de4f7cd780ba367cccbc2593f38d18ce6df89/instposhsshdev")

And the script itself:

$userName = "USER_NAME"
$credentials = New-Object -TypeName System.Management.Automation.PSCredential ($userName, (new-object System.Security.SecureString))
$sshKey = "SSH_KEY_PATH"
$localFolder = "LOCAL_FOLDER_PATH"
$remoteFolder = "REMOTE_FOLDER_PATH"
$computerName = "MACHINE_ADDRESS"
$port = 22
$commands = @("CMD_1", "CMD_2", "CMD_3")

Write-Host "Uploading data from $localFolder to $remoteFolder on $userName|$computerName|$port" -ForegroundColor Yellow
Set-SCPFolder -LocalFolder $localFolder -RemoteFolder $remoteFolder -ComputerName $computerName -Port $port -AcceptKey -Credential $credentials -KeyFile $sshKey
Write-Host "Done" -ForegroundColor Yellow

Write-Host "Connecting to $userName|$computerName|$port to execute commands" -ForegroundColor Yellow
$session = New-SSHSession -ComputerName $computerName -Port $port -AcceptKey -Credential $credentials -KeyFile $sshKey
ForEach ($cmd in $commands) {
Write-Host "Executing $cmd" -ForegroundColor Yellow
$result = (Invoke-SSHCommand -SSHSession $session -Command $cmd).Output
Write-Host $result.split("`n")
}

Remove-SSHSession -SSHSession $session

This code does following:

  1. It copies folder LOCAL_FOLDER_PATH into REMOTE_FOLDER_PATH on target machine via SCP (and SSH key)
  2. Next it connects via SSH (key) and executes commands CMD_1, CMD_2 and CMD_3.

 

So now, after each new build I can run this script and a new version of the project will be uploaded to the machine. Or this script can be set as a post build task into pipeline.

Uncategorized

Meme Code Review Google Chrome Extension

Introduction

Code review can be sometimes very boring. Checking hundreds of lines can be more fun with memes in comments!
For this purpose I made a small extension into Google Chrome that enables you to insert meme in a simple way.

Current version supports this feature on GitHub and Bitbucket.

How to use it

After installation of the plugin, you have to set memes that you want to use (menu -> settings -> extensions -> Meme Code Review -> options). On this page you set name and url of the meme image (see Figure 1).

Figure 1
Figure 1

Now, you can use it -> just open GitHub (Figure 2) or Bitbucket (Figure 3). Then click on the troll button. A list of saved memes appears. If you click on the meme image, its address will be inserted into text area.

Enjoy it!

Source can be found on https://github.com/marazt/meme-code-review.

Figure 2
Figure 2
meme3
Figure 3

 

Uncategorized

Setup Wifi Autoconnect to Hidden Network on Raspberry PI2

Introduction

Because I wanted to set my Raspberry to the different location that is router, I have added a USB Wifi dongle (TP-LINK TL-WN725N v2) into it. Last version of Raspbian Jessie has drivers for it so it is no problem. Next, there is described to make it working on Raspberry wiki.

The problem is that it does not work with hidden and secured Wifi :/.

 

Let’s Make It Working

  • Edit /etc/wpa_supplicant/wpa_supplicant.conf
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Add following code at the end:

network={
     ssid="***HOTSPOT NAME***"
     proto=RSN
     key_mgmt=WPA-PSK
     pairwise=CCMP TKIP
     group=CCMP TKIP
     psk="***HOTSPOT PASSWORD***"
     scan_ssid=1
}
  • Edit /etc/network/interfaces
sudo nano /etc/network/interfaces

Change line auto lo into auto wlan0

  • Restart wifi
sudo ifdown wlan0
sudo ifup wlan0
  • Now is should connect to hotspot. You can test it via command
ifconfig wlan0

It should print something like this:

pi@raspberrypi:~ $ ifconfig wlan0
wlan0     Link encap:Ethernet  HWaddr 60:e3:27:12:05:4d
     inet addr:10.0.0.45 Bcast:10.0.0.255  Mask:255.255.255.0
     inet6 addr: 2a00:1028:838e:ee:3b84:bf0d:3bb:875e/64 Scope:Global
     inet6 addr: fe80::8afd:f970:f42e:29ff/64 Scope:Link
     UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
     RX packets:473 errors:0 dropped:47 overruns:0 frame:0
     TX packets:321 errors:0 dropped:1 overruns:0 carrier:0
     collisions:0 txqueuelen:1000
     RX bytes:65018 (63.4 KiB)  TX bytes:47021 (45.9 KiB)

attribute inet addr should show IP address that has Raspberry received.

  • Now try to restart it and check again that is has connected
Uncategorized

Visual Studio Team Services & Custom Build & Test Results

Introduction

If you have Visual Studio solution, you have no problem to setup build definition – you just select Visual Studio Test task and Publish Test Results task. You don’t have to configure it additionally – it works ok with default configuration.

But if your project is not VS solution, you have to run your own script to run tests. Next problem can be unit test framework – VSTests does not exists for non .NET world, they use mainly JUnit, XUnit or NUnit.

All this is possible in VSTS

How to Setup Custom Unit Tests and Results

  • Create new build definition and select Empty definition (see Figure 1).
Figure 1
Figure 1
  • Add new Utility task(s), e.g. Batch or Shell script (depending on your system and build agent) – see Figure 2. Set path to file that should be executed. In this file(s) should be set scripts that, e.g., download packages needed by your application, and of course run tests.

For example, is can look like this:


# prerequisites: pip, virtualenv in path
# install new virtual environment and download dependencies

virtualenv venv

source venv/bin/activate

pip install -r requirements.txt


# run unit tests

nosetests tests --with-xunit --xunit-file=TEST-results.xml

As you can see, unit tests are run with nosetests (unit test framework for python) and test results are generated as XUnit format.

git2
Figure 2

 

  • Add Publish Test Results task (Figure 3) and setup it (Figure 4):
git3
Figure 3
Figure 4
Figure 4

Important fields are Test Result format, Test Result Files and Enabled checkbox. For first simple setup it is enough. Save the definition.

Note: you can see that nosetests should generate XUnit file, but in publish task is set JUnit, not XUnit option. This is correct. In real, nosetests generate JUnit file. It is strange, but true.

Now run the build (in my case on custom build agent running on Ubuntu VM in Azure) and see results (Figure 5).

Figure 5
Figure 5

So even you are not using VS solution, .NET language or Windows, you have working unit tests running on VSTS!

 

 

Uncategorized

Full DevOps Process With Visual Studio Team Services & GitHub or other Git source

Introduction

If you host your code on VSTS, you have no problem to:

commit code -> automatic test run -> deployment -> integration & load tests, …

But what if you have your code not hosted on VSTS, e.g. on GitHub?

I have been solving this problem via pushing into two repositories – first to VSTS to run tests, second to push code to public GitHub repository.

But now it is possible to use different source code repository, not only from VSTS (see Figure 1). Just select option and set credentials.

Figure 1
Figure 1

Here is a good tutorial how to use GitHub code directly from MS. This tutorial can be used for other sources too.

 

 

 

 

Uncategorized

Code Completion Version 1.0.0.10 Released

Code Completion version 1.0.0.10 has been released.

It contains fix of the strange behavior after migration to VS2015 project and new  manifest version (vsx-schema 2.0.0.0). Although the extension was correctly loaded into IDE, all controls and dialog windows were working, functionality itself does not.

I have migrated the manifest back to version 1.0.0.0 which helped. It is strange because my other extension works well with the new manifest.

Uncategorized

How To Copy Images/VHDs Between Azure Locations

Introduction

To be able to establish same virtual machine in different location, you have to do following steps:

  • stop VM
  • create copy of the VM
  • move this image to new location
  • create a disk from copied image
  • create a new VM from this disk.

Backuping can be done via Azure Portal, but moving of the image is not possible. But there exist PowerShell scripts. In this post is script which copies VM image from source to target location. Full script for other steps will be in another post.

Script

# Select your Azure account
Select-AzureSubscription "Azure account"

# URI of the image/vhd to be copied
$sourceImage = "uri of the image/vhd to be copied"

# Source storage account
$sourceStorageAccount = "source-storage-account"
$sourceStorageKey = "source-storage-key"

# Target storage account
$targetStorageAccount = "target-storage-account"
$targetStorageKey = "target-storage-key"

# Set target container
$targetContainderName = "copied-vhd"

# Set targt blob
$targetBlobName = "copied-vhd"

# Create the source storage account context
$sourceContext = New-AzureStorageContext  –StorageAccountName $sourceStorageAccount `
-StorageAccountKey $sourceStorageKey

# Create the target storage account context
$targetContext = New-AzureStorageContext  –StorageAccountName $targetStorageAccount `
-StorageAccountKey $targetStorageKey

# Create container in taret destination
New-AzureStorageContainer -Name $containerName -Context $destContext

# Start the asynchronous copy
$blobCopy = Start-AzureStorageBlobCopy -srcUri $sourceImage `
-SrcContext $sourceContext `
-DestContainer $containerName `
-DestBlob $targetBlobName `
-DestContext $targetContext

# Print status while it is not done
$status = $blobCopy | Get-AzureStorageBlobCopyState
$status

While($status.Status -eq "Pending"){
$status = $blobCopy | Get-AzureStorageBlobCopyState
Start-Sleep 15
cls
Get-Date -format "HH:mm:ss yyyy-MM-dd"
$status
}

EDIT: I have found good article with full migration script >here<.

 

 

Uncategorized

Angular & html5Mode & IIS

Introduction

If you want to have pretty URLs of your Angular website ( http://www.example.org/contact instead of http://www.example.org/#/contact), you can do it by setting $locationProvider.html5Mode(true). See docs for more information.

But now, there occurs following problem – if you run your application, set address directly to browser, e.g. http://www.example.org/contant and press enter, it returns error 404. It is because this page (file) does not exist.

So you have to update configuration of your site to rewrite URL just to root, e.g. http://www.example.org/. Specific routing is done by Angular itself.

 

How To Do It

  • Install URL Rewrite module into IIS

This module is not part of IIS default installation. You can download it here.

  • Update site configuration

Open IIS console. There should be new icon URL Rewrite (see Figure 1).

 

Figure 1
Figure 1

You can use GUI to set it or you can update site web.config file directly. Just add following section into system.webServer element in site web.config:


<rewrite>
<rules>
<rule name="AngularJS" stopProcessing="true">
<match url=".*" />
<conditions logicalGrouping="MatchAll">
<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
<add input="{REQUEST_URI}" pattern="^/(api)" negate="true" />
</conditions>
<action type="Rewrite" url="/" />
</rule>
</rules>
</rewrite>

 

Now restart site and routing should work as expected!

Uncategorized

Setup Own Build Agent For Visual Studio Team Systems

Introduction

By default, Visual Studio Team Systems (aka Visual Studio Online) provides Hosted Build Agent for free (~ 240 minutes of CPU time by month).

This agent runs on Windows machine and has no problem to run your VS projects, batch files, command-lines command, etc.

But there is no agent running on Linux/OSX machine. Because I have projects which I want to be run on these platforms, e.g. RestMocker, I had to establish own agent.

You can use one own for free, for additional agents you have to pay. Microsoft provides good tutorial on GitHub how to install this agents on supported platforms.

 Let’s Go To Work

 Setup Azure virtual machine with Ubuntu

I have established my virtual machine (Ubuntu 15.04) on Azure. But you can setup it where you want (Digital Ocean, Google, …).

Agent Setup

  • Log in
  • To be sure, run following scripts

# update and upgrade repositories

sudo apt-get update

sudo apt-get upgrade

# install nodejs

sudo apt-get install node # (on Ubuntu run sudo apt-get install nodejs)

# on Ubuntu create symlink to able to use node command

sudo ln -s `which nodejs` /usr/bin/node

# install npm

sudo apt-get install npm
  • Download latest installer

sudo npm install vsoagent-installer -g

sudo chown -R $USER ~/.npm

  • Create directory for agent

mkdir myagent
cd myagent
vsoagent-installer

  • Allow agent to listen on your VSTS

In your VSTS select Settings. Go to Control Panel and select tab Agent pools (see Figure 1).

Figure 1
Figure 1

 

In All pools -> Default there should be no agent now. Switch to Roles.

There should be two roles (Agent Pool Administrators, Agent Poll Service Accounts) – see Figure 2.

Figure 2
Figure 2

Select Agent Poll Service Accounts. You should see two account in this role (Agent Poll Service Accounts, Default – Agent Service).

To be able to listen with our new agent, add new account. Press + and set your account email. You account should be added (see Figure 3).

Figure 3
Figure 3

 

  • Run agent for first time

node agent/vsoagent

You will be asked to set agent name, server where to listen, e.g. https://nickname.visualstudio.com, your account name and password.

Now, you should see your agent in VSTS with green label (Figure 4). When you disconnect agent or it cannot log in, label next agent will be red (Figure 5).

4_1after_first_connection - Copy
Figure 4
Figure 5
Figure 5

 

  • Run agent as service

Agent itself supports running as a service only on OSX, Linux is not supported now.

So, let’s run the agent in nohup:


/usr/bin/nohup node agent/vsoagent -u ACCOUNT -p PASSWORD > nohup.log &

Yes, it is not secured as well, so be careful setting your credentials this way.

  • Let’ try it

Now, go to your project build definition. Here select one of your builds which is capable run on Linux machine and select Queue Build. In dialog, select Default Queue and press Ok (see Figure 6).

Figure 6
Figure 6

That’s all. Now your builds should run on your private agent.

Uncategorized

How to Establish Psycopg2 on OSX

Because my Lenovo X1 Carbon died this week (what the hell – non working adaptive keyboard, ghost screen effect and finally damaged mother-board !!!), I have to work with iMac a few days because there was no other Win pc in the office (And it is very painful and hard task if you want to be productive).

Mono is working correctly, Python is working correctly, but as on Windows, psycopg2 is NOT working correctly after installation and there must be done additional steps to make it working.

Note: PostgreSql must be installed. In this tutorial I have installed last version 9.4.

Possible problems:

  • First when I tried pip install psycopg2, there was following error:

Error:  pg_config executable not found.

Solution: activate virtualenv you want to use and run following command. It adds path to PostgreSql bin folder where is situated pg_config file.

PATH=$PATH:/Library/PostgreSQL/9.4/bin/
  • When I run project where was psycopg2 imported, I got:

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so Library libssl.1.0.0.dylib Library libcrypto.1.0.0.dylib

Solution: run following commands to create symlinks to needed libs. If the symlinks already exist, remove them first (or rename them to have backup).

# first make backup
sudo mv /usr/lib/libssl.1.0.0.dylib /usr/lib/libssl.1.0.0.dylib.old
sudo mv /usr/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.1.0.0.dylib.old
# create new symlinks
sudo ln -s /Library/PostgreSQL/9.4/lib/libssl.1.0.0.dylib /usr/lib/libssl.1.0.0.dylib
sudo ln -s /Library/PostgreSQL/9.4/lib/libcrypto.1.0.0.dylib /usr/lib/libcrypto.1.0.0.dylib
  • Then I run the app again and obtained:

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Symbol not found: _lo_lseek64 Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so Expected in: /usr/lib/libpq.5.dylib in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so

Solution: Run following commands to backup actual symlink and make correct one.

# first make backup
sudo mv /usr/lib/libpq.5.dylib /usr/lib/libpq.5.dylib.old
# create new symlinks
sudo ln -s /Library/PostgreSQL/9.4/lib/libpq.5.dylib /usr/lib

And now, it should work correctly!