Search This Blog

2026-09-25

PowerShell - Getting Remote MS SQL Server 2017 SSL Certificate Using Low Level Network Communication

Product: Microsoft MS SQL Server
Version: 2017

Overview

There are few scenarios where you would like to download the SSL certificate and save as file while working with MS SQL Server client or server programs.  I come into situation where the old version of IBM Cognos Analytics required to add remote MS SQL Server's self-signed SSL cert into its own Java keystore.  The challenge is that the MS SQL Server does not has any self-signed SSL cert, and it dynamically creates a SSL cert (MS called it as fallback SSL cert) in memory during MS SQL Server startup.  So it is not possible to save the certificate file.

Following are few more scenarios where you might want to save the remote MS SQL Server's SSL cert:

  1. MS SQL Server DBA does not has Windows admin privilege to run Windows Certificate Manager (certmgr.msc) to save the SSL cert
  2. Windows admin is not familiar with Windows Certificate Manager (certmgr.msc) and not able to provide the SSL cert
  3. MS SQL Server DBA does not know anything about SSL fallback cert, when MS SQL Server does not has any SSL cert

The MS SQL Server API Microsoft.Data.SqlClient in any programming, e.g. Visual C#, Visual C++, Java, Python, Perl, is not possible to obtain the SSL cert.

openssl.exe utility has drop MS SQL Server TDS (Tabular Data Stream) support, so it is not possible to use this utility to get the SSL cert.

As the result, I develop a PowerShell program that uses low level TCP network communication that can download SSL certificate from local or remote MS SQL Server through TCP/IP protocol.

Technical Design

Low level TCP programming uses PowerShell System.Net.Sockets.TcpClient.  MS SQL Server login is always SSL encrypted.  It contains 2 phase, which is called PRELOGIN and LOGIN7.  During PRELOGIN phase, MS SQL Server will provide SSL certificate to the DB client program, which then can proceeds with LOGIN7 phase that will login to MS SQL Server with username and password (assumes DB authentication, not Windows user).

The most challenging part is research MS SQL Server PRELOGIN protocol, as this is poorly documented by Microsoft.  You can find the official document from learn.microsoft.com below:

Title: Open Specifications > Protocols > SQL Server Protocols > Technical Documents > [MS-SSTDS]: Tabular Data Stream Protocol Version 4.2 > Messages > Message Syntax > Packet Header Message Type Stream Definition > PRELOGIN

  • Ref: https://learn.microsoft.com/en-us/openspecs/sql_server_protocols/ms-sstds/75e62f67-f057-4d46-82b3-6920fe0ebada

The PowerShell program will make use of hex and binary to convert the PRELOGIN data structure back and forte with remote MS SQL Server server.

While reading MS' document, we need to ignore any document that tell encryption is optional, as well as TLS v1.2 is optional.  In fact, many versions of MS SQL Server have long enforced TLS v1.2, and encryption during login.  So there are many obsolete content which confuses reader like us.  The login sequence is called as Tabular Data Stream with short name of TDS.  My program is written for MS SQL Server 2017, which is using TDS v7.4, but newer MS SQL Server uses TDS v8.0.  So this program might not work with newer MS SQL Server due to a change in PRELOGIN phase.

My program set the encryption as 0x01, which in MS' doc called "Encryption is available and on."  Again, all MS SQL Server support SSL cert, so you can safely accept encryption is available.

Some variables in my program call "stream" which represent TCP network communication.  At the beginning of the low level TCP network connection, a variable of NetworkStream is created against TcpClient connection.  This is to facilitate back and fort communication with remote MS SQL Server.

Soon after open TCP connection, my program will send a static PRELOGIN to remote MS SQL Server to tell it that my program needs to login.  This part of the communication is not encrypted, and in clear text. Don't simply change this static PRELOGIN array value, as 2 of the bytes (2 array elements) contains number of bytes inside this array.  You can refer to MS doc for exact array element, if you want to change this array.

After that, MS SQL Server will reply back to tell my program its MS SQL Server version, whether it supports encryption (it will be always supported), number of bytes it will send back.  My program will retrieve this reply (MS doc called message or message stream) using 2 separate steps (command NetworkStream.Read()).  The reason is because the 1st part is a constant 8 bytes, which I called "payload header" while MS doc implies it is payload as well, with short name of PL.  My program then find out number of bytes remote MS SQL Server going to send to me, and in the 2nd part, my program will read the remaining reply (message stream).  I called the 2nd part of it as "prelogin payload option token" and "prelogin payload data."  MS doc is not clear about how they called them.  For my program, I mainly determine the remote MS SQL Server will tell my program that it accept my PRELOGIN request, and it does support SSL encryption.  This is very important as SSL cert will only be send to my program if it replied that it supports SSL encryption.  This will double confirm new MS SQL Server standard that LOGIN7 is always SSL encrypted.

The next part is the most challenging part as I need to continue to talk to MS SQL Server using TDS header structure, while enforcing SSL TLS v1.2.  For this part, I can't hard code the static array like earlier as I need to call System.Net.Security.SslStream.AuthenticateAsClient which will indirectly calls System.IO.Stream over TCP.  So there is an extra class called TdsTlsStream that helps me to perform read() and write() over the TCP network communication.  Inside this class, in read() function, it will read first 8 bytes to determine network packet length, and in write() function, it will create extra 8-bytes packets to insert before sending over the data to remote MS SQL Server.

System.Net.Security.SslStream.AuthenticateAsClient is the main call that will obtain SSL certificate from remote MS SQL Server.  Anything that is wrong in the program will leads to timeout, then remote MS SQL Server will close the network connection.  If it success, then the program will be able to extract the SSL certificate using System.Security.Cryptography.X509Certificates.X509Certificate2.

You can further modify the program to add following line to save the SSL certificate as a file locally:

E.g. Export-Certificate -Cert $<yr variable name> -FilePath db_certificate_DER_binary.cer -Type CERT

Above command will create a new SSL certificate file called db_certificate_DER_binary.cer in current directory

PowerShell Program

Following is the PowerShell program

[CmdletBinding()]
param()


# Define your SQL Server instance details, default port 1433
$SqlServer = "your-ms-sql-server-hostname-or-IP"
$SqlPort   = 20277


# Enforce TLS 1.2 for the handshake protocol selection
$TlsVersion = [System.Security.Authentication.SslProtocols]::Tls12
# --- SQL Server Pre-Login Handshake Protocol Payload ---
# This standard 8-byte TDS header tells SQL Server a client login attempt is starting.
[byte[]]$TdsPreLoginPacket = 0x12, 0x01, 0x00, 0x2F, 0x00, 0x00, 0x01,0x00, 0x00, 0x00, 0x1A, 0x00, 0x06, 0x01,0x00, 0x20,0x00, 0x01, 0x02, 0x00, 0x21, 0x00, 0x01,0x03, 0x00, 0x22, 0x00, 0x04, 0x04, 0x00,0x26, 0x00,0x01, 0xFF, 0x09, 0x00, 0x00, 0x00, 0x00,0x00, 0x01, 0x00, 0xB8, 0x0D, 0x00, 0x00,0x01
 
# Create a TCP Client to connect to the SQL port
$TcpClient = New-Object System.Net.Sockets.TcpClient
try {
    $TcpClient.Connect($SqlServer, $SqlPort)
$TcpClient.SendTimeout    = 10000;
$TcpClient.ReceiveTimeout = 10000;

    $NetworkStream = $TcpClient.GetStream()

    $NetworkStream.Write($TdsPreLoginPacket, 0, $TdsPreLoginPacket.Length)


# 3. Read the server's TDS Pre-Login Response Header
$responseHeader = New-Object byte[] 8
$bytesRead = $NetworkStream.Read($responseHeader, 0, $responseHeader.length)
if ($bytesRead -lt $responseHeader.length) {
Write-Error "PRELOGIN - $bytesRead bytes read, but expects 8 bytes"
throw "Failed to read a valid PRELOGIN TDS header response"
}

$respPayloadLength = ($responseHeader[2] -shl 8) + $responseHeader[3] - $responseHeader.length

Write-Host "--- TDS HEADER RECEIVED ---" -ForegroundColor Cyan
Write-Host "Raw Bytes Header: " ($responseHeader -join ' ')
Write-Host "Remaining Payload Length: $respPayloadLength bytes" -ForegroundColor Cyan

    # Continue to read the token data and data
if ($respPayloadLength -gt 0) {
$responsePayload = New-Object byte[] $respPayloadLength
$bytesRead = $NetworkStream.Read($responsePayload, 0, $respPayloadLength)
$intEndofTokenHeader = [array]::IndexOf($responsePayload, [byte]0xff)
$intEndofTokenData = $intEndofTokenHeader + 1
Write-Host "Finished reading payload: $respPayloadLength bytes" -ForegroundColor Cyan
Write-Host "Offset of FF terminator: " $intEndofTokenHeader -ForegroundColor Cyan
Write-Host "Raw Bytes: " ($responsePayload -join ' ') -ForegroundColor Gray
Write-Host "Raw Bytes Token Header: " ($responsePayload[0..$intEndofTokenHeader] -join ' ') -ForegroundColor Gray
Write-Host "Raw Bytes Token Data: " ($responsePayload[$intEndofTokenData..($respPayloadLength - 1)] -join ' ') -ForegroundColor Gray
}

$isServerEncrypted = 0
$posPayload = 0
while ($posPayload -lt $intEndofTokenHeader) {
$plOptionType = $responsePayload[$posPayload]
Write-Host "PRELOGIN respond header - Position $posPayload=$plOptionType"
# Exit loop if reach header terminator character 0xFF
if ($plOptionType -eq 0xFF) {
Write-Host "PRELOGIN respond header - Reached end of header"
break
}
elseif ($plOptionType -ne 1) {
$posPayload += 5
Write-Host "PRELOGIN respond header - Skipping 5 bytes to offset $posPayload"
continue
}

        # Calculate the address of encryption byte
$valueOffset = [System.BitConverter]::ToUInt16([byte[]]@($responsePayload[$offset + 2], $responsePayload[$offset + 1]), 0)
Write-Host "PRELOGIN respond header - Found encryption byte in offset address $posPayload with value $valueOffset"
$isServerEncryptedRaw = $responsePayload[$valueOffset]
$isServerEncrypted = $isServerEncryptedRaw -band 0x0F
Write-Host ("PRELOGIN respond header - offset $valueOffset's encryption byte value = {0} (0x{0:X2}) -> {0} (0x{1:X2})" -f $isServerEncrypted,$isServerEncrypted ) -ForegroundColor Cyan
break
}

Write-Verbose "Entryption is supported, so it is possible to get SSL cert. $isServerEncrypted is not 0 means DB server will perform encryption"
Write-Verbose "Setup TLS stream"
$tlsStreamToWrap = [TdsTlsStream]::new($NetworkStream)

    # Wrap the network stream in an SslStream object for the TLS handshake
# We pass a callback to automatically trust self-signed SQL dev certificates
Write-Verbose "Setup SSL stream inside TLS stream"
    $SslStream = New-Object System.Net.Security.SslStream($tlsStreamToWrap, $false, {
        param($sender, $certificate, $chain, $sslPolicyErrors)
        # Always return true to capture the certificate even if untrusted/self-signed fallback
        return $true
    })

    Write-Host "SslProtocol negotiated #1: $($sslStream.SslProtocol)"

    # Authenticate as client to trigger the handshake
Write-Verbose "Specified SSL to use $TlsVersion, and obtain remote DB server's SSL cert"
    $SslStream.AuthenticateAsClient($SqlServer, $null, $TlsVersion, $false)
    Write-Host "SslProtocol negotiated #2: $($sslStream.SslProtocol)"

    # Extract the server certificate
$ServerCertificate = [System.Security.Cryptography.X509Certificates.X509Certificate2]($SslStream.RemoteCertificate)

    if ($ServerCertificate) {
        Write-Host "`n=== SQL Server TLS 1.2 Certificate Details ===" -ForegroundColor Green
        $ServerCertificate | Format-List Subject, Issuer, SerialNumber, Thumbprint, NotBefore, NotAfter
        # Verify if it is the fallback certificate
        if ($ServerCertificate.Subject -like "*SSL_Self_Signed_Fallback*") {
$Host.PrivateData.VerboseForegroundColor = 'Cyan'
            Write-Verbose "This is the internal SQL Server Self-Signed Fallback Certificate, so it is in RAM, and will change after DB server start/restart/reboot"
        }
    } else {
        Write-Warning "Successfully connected with TLSv1.2, but no remote certificate was returned by the server."
    }


} catch {
$lineText   = $_.InvocationInfo.Line
    $lineNumber = $_.InvocationInfo.ScriptLineNumber
    $position   = $_.InvocationInfo.PositionMessage

    Write-Host "❌ Error occurred on line $lineNumber" -ForegroundColor Red
    Write-Host "Code block: $lineText" -ForegroundColor Yellow
    Write-Host "Position details:`n$position"
} finally {
    # Properly close and dispose of connections
    if ($SslStream) { $SslStream.Dispose() }
    if ($TcpClient) { $TcpClient.Close(); $TcpClient.Dispose() }
}



class TdsTlsStream : System.IO.Stream {
[System.IO.Stream]$InnerStream
[int]$PayloadLength = 0

TdsTlsStream([System.IO.Stream]$InnerStream) {
$this.InnerStream = $InnerStream
}

[bool] get_CanRead() { return $this.InnerStream.CanRead }
[bool] get_CanWrite() { return $this.InnerStream.CanWrite }
[bool] get_CanSeek() { return $this.InnerStream.CanSeek }
[Int64] get_Length() { return $this.InnerStream.Length }
[Int64] get_Position() { return $this.InnerStream.Position }
[void] set_Position([Int64]$Value) { $this.InnerStream.Position = $Value }
[int] get_ReadTimeout() { return $this.InnerStream.ReadTimeout }
[int] get_WriteTimeout() { return $this.InnerStream.WriteTimeout }

[void] Flush() { $this.InnerStream.Flush() }
[Int64] Seek([Int64]$Offset, [System.IO.SeekOrigin]$Origin) { return $this.InnerStream.Seek($Offset, $Origin) }
[void] SetLength([Int64]$Value) { $this.InnerStream.SetLength($Value) }

[int] Read([byte[]]$Buffer, [int]$Offset, [int]$Count) {
# Removes TDS header before setting the network Buffer
if ($this.PayloadLength -eq 0) {
$tlsHeader = [byte[]]::new(8)
$read = 0
while ($read -lt 8) {
$read += $this.InnerStream.Read($tlsHeader, 0, 8)
}

$lengthBeforeHeader = [System.BitConverter]::ToUInt16([byte[]]@($tlsHeader[3], $tlsHeader[2]), 0)
$lengthBeforeHeader -= 8
$this.PayloadLength = $lengthBeforeHeader
}

if ($Count -gt $this.PayloadLength) {
$Count = $this.PayloadLength
}
$read = $this.InnerStream.Read($Buffer, $Offset, $Count)
$this.PayloadLength -= $read
return $read
}

[void] Write([byte[]]$Buffer, [int]$Offset, [int]$Count) {
$newPayload = $this.GenerateTdsHeader($Buffer, $Offset, $Count)
$this.InnerStream.Write($newPayload, 0, $newPayload.Length)
}

[byte[]] GenerateTdsHeader([byte[]]$Payload, [int]$Offset, [int]$Count) {
# The length is big endian encoded so it is inserted in reverse order
$lengthBytes = [System.BitConverter]::GetBytes([uint16]($Count + 8))

$newPayload = [byte[]]::new(8 + $Count)
$newPayload[0] = 0x12  # Type - Pre-Login
$newPayload[1] = 0x01  # Status - End of message (EOM)
$newPayload[2] = $lengthBytes[1]
$newPayload[3] = $lengthBytes[0]
$newPayload[4] = 0  # SPID
$newPayload[5] = 0  # SPID
$newPayload[6] = 0  # PacketID
$newPayload[7] = 0  # Window
[System.Array]::Copy($Payload, $Offset, $newPayload, 8, $Count)

return $newPayload
}
}

2026-04-01

SAP Data Services Web Service (SOAP) Calling - Example

Product: SAP Data Services
Version: 4.2.x - 4.3.x

Overview

SAP BODS (Data Services) has DSMC browser GUI which offers web service functionality.  This allows external scheduler to easily integrate with it.

This blog post is going to show using PowerShell Invoke-WebRequest, so that you can see its  SOAP communication in low level.  For experience person, it should be easy to figure out how to configure the external job scheduler to use BODS WSDL 2.1 web service (SOAP) call.

For this example, I'm using following dummy value to illustrate

  • BODS URL: http://localhost:8080/DataServices/servlet/webservices?ver=2.1
  • Server hostname: localhost
  • Authentication Method: Build in Enterprise
  • Username: administrator
  • Password: hidden_hash_value
  • Job Server local repository name: LOCAL_REPO_DEV1
  • Job ID: 2
  • Going to simulate the OOTB web service call Get_BatchJob_Status, which will return 2 values of whether the web service call success or not (0=success), as well as job status in string of Running/Succussed/Error/Warning
    • BODS doc: https://help.sap.com/docs/SAP_DATA_SERVICES/ab33122a997f40d89e340549ff0bced8/5748962b6d6d1014b3fc9283b0e91070.html
Login is required to obtain session ID.  The session ID going to be used in subsequent web service call

For full SAP BODS web service reference doc, see https://help.sap.com/docs/SAP_DATA_SERVICES/ab33122a997f40d89e340549ff0bced8/574996556d6d1014b3fc9283b0e91070.html

Using PowerShell Invoke-WebRequest to Call BODS Web Service

1. Login

$headers1 = @{"SOAPAction" = "function=Logon"}

$soapBody1 = @"

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://www.businessobjects.com/DataServices/ServerX.xsd">

   <soapenv:Header/>

   <soapenv:Body>

      <ser:LogonRequest>

         <username>administrator</username>

         <password>hidden_hash_value</password>

         <cms_system>localhost</cms_system>

         <cms_authentication>secEnterprise</cms_authentication>

      </ser:LogonRequest>

   </soapenv:Body>

</soapenv:Envelope>

"@


$response1 = Invoke-WebRequest -UseBasicParsing -Method Post -ContentType "text/xml;charset=UTF-8" -Headers $headers1 -Body $soapBody1 http://localhost:8080/DataServices/servlet/webservices?ver=2.1

[xml]$xmlResponse1 = $response.Content

$dsmc_SessionID = $xmlResponse1.Envelope.Body.session.SessionID

$dsmc_SessionID


2. Call Get_BatchJob_Status

$headers2 = @{"SOAPAction" = "jobAdmin=Get_BatchJob_Status"}

$soapBody2 = @"

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://www.businessobjects.com/DataServices/ServerX.xsd">

   <soapenv:Header>

      <ser:session>

         <SessionID>$dsmc_SessionID</SessionID>

      </ser:session>

   </soapenv:Header>

   <soapenv:Body>

      <ser:batchJobStatusRequest>

         <runID>2</runID>

         <repoName>LOCAL_REPO_DEV1</repoName>

      </ser:batchJobStatusRequest>

   </soapenv:Body>

</soapenv:Envelope>

"@


$response2 = Invoke-WebRequest -UseBasicParsing -Method Post -ContentType "text/xml;charset=UTF-8" -Headers $headers2 -Body $soapBody2 http://localhost:8080/DataServices/servlet/webservices?ver=2.1

[xml]$xmlResponse2 = $response2.Content

$xmlResponse2.Envelope.Body.batchJobStatusResponse


localtypes                                              returnCode status
----------                                              ---------- ------
http://www.businessobjects.com/DataServices/ServerX.xsd 0          error

Using PowerShell New-WebServiceProxy to Call BODS Web Service

This is using SOAP web service call.

$url_dsmc = "http://localhost:8080/DataServices/servlet/webservices?ver=2.1&wsdlxml"
# First call to BODS web service server to get the wsdl
$response1 = New-WebServiceProxy -Uri $url_dsmc -Namespace dsmc_namespace

$dsmc_namespace = $dsmc.GetType().Namespace
$dsmc_conn_ops = New-Object -TypeName "$dsmc_namespace`.Connection_Operations"

$param = New-Object "dsmc_namespace.LogonRequest"
$param.username = "administrator"
$param.password = "hidden_hash_value"
$param.cms_system = "localhost"
$param.cms_authentication = "secEnterprise"

# 2nd call - Triggers logon, and it will return Session ID
# Stores the session_id for next call
$session_id = $dsmc_conn_ops.Logon($param)

# Next is to check Job status
# Local repo: LOCAL_REPO_DEV1
# Job's run ID 2
$job_request = New-Object "dsmc_namespace.batchJobStatusRequest"
$job_request.repoName = "LOCAL_REPO_DEV1"
$job_request.runID = 2
$dsmc_Batch_Job_Admin = New-Object -TypeName "$dsmc_namespace`.Batch_Job_Admin"
$dsmc_Batch_Job_Admin_namespace = $dsmc_conn_ops.GetType().Namespace
$dsmc_Batch_Job_Admin.sessionValue = $session_id

# 3rd time calling BODS SOAP web service
$dsmc_Batch_Job_Admin.Get_BatchJob_Status($job_request)

Output
returnCode status
---------- ------
         0 error

Analysis

The parameter and web service methods required to referring to BODS web service wsdl.  For example:
  1. New-Object "dsmc_namespace.LogonRequest" - wsdl contains "<xsd:element name="LogonRequest">" with complexType, and 4 string parameters of username , password , cms_system , cms_authentication 
  2. Logon() - wsdl indicated that the pass-in parameter is call "LogonRequest" directly under the root. It does not belongs to any class (PowerShell), or wsdl:binding (wsdl)
  3. PowerShell needs to call New-Object -TypeName $dsmc_namespace`.<wsdl:binding> to switch between 5 different binding of Batch_Job_Admin, Repo_Operations, Batch_Jobs, Connection_Operations, Realtime_Service_Admin (default)

2025-12-21

FreeNAS, TrueNAS, FreeBSD - Shrink ZFS File System

Product: Debian distribution, TrueNAS, FreeNAS

Version: Any version which uses ZFS File System

Overview

ZFS always create mount point as volume group equivalent volume even you assigns 1 partition to a mount point.  In Debian, TrueNAS or FreeNAS, it always creates as volume group (ZDEV), regardless you assigns entire hard drive (1 device) or 1 partition to it.

ZFS does not has build-in functionality to shrink partition, but we can remove partition after adding another one.  So the strategy to shrink partition is below:

  1. In the first disk, which contains the partition that we would like to shrink, finds out how much space is used by the mount point or partition.  For example, let's assume 8GB is used in a partition which is 500GB
  2. Locates another disk or partition which can hold all the data used by above partition.  We will later leverage ZFS feature to auto-copy its content over to this new partition.  For simplicity, let's call this as 2nd disk
  3. In the 2nd disk, creates a partition which is bigger than 8GB.  For example, creates a new 16GB partition
  4. Unmount the mount point to avoid any user copying any data into the partition which we are going to shrink
  5. Adds the new 16GB partition from 2nd disk into the same partition (volume, or ZDEV) which we would like to shrink.  Don't worry that we are making the volume 500GB + 16GB larger.  ZFS will not copy anything into this volume since we unmount it
  6. Detach the partition we would like to shrink from the volume (ZDEV) to force ZFS to copy all the used data from this partition into 2nd disk's new partition (16GB partition size)
  7. Now, you can delete and re-create the partition with smaller size in 1st disk
  8. Adds back this partition into the volume or ZDEV.  Do not mount the mount point yet
  9. Detach the 2nd disk's 16GB partition from the volume, so what ZFS will copy all the content to the newly added but smaller partition
  10. This complete the partition shrink

Procedure

Assumes the partition we would like to shrink is called boot-pool (FreeNAS' name for boot volume). It contains partition sda3 inside disk 1 (device name called sda).  The 2nd partition name that going to temporary add is called vg00/zvol0.

  1. Checks how much space is used in boot-pool
    1. Command: zpool list -v boot-pool

% zpool list -v boot-pool
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
boot-pool   928G  3.18G   925G        -         -     0%     0%  1.00x    ONLINE  -
  sda3      931G  3.18G   925G        -         -     0%  0.34%      -    ONLINE
  1.  Above shown 3.18GB is used.  It contains only 1 partition called sda3 in disk sda
  2. Locates a device which is bigger than 3.18GB.  Let's say 8GB
    1. Option 1: If TrueNAS, then create a new Zvol from any of the VDEV (storage pool)
    2. Option 2: Creates a new partition sdb1, sdb2, or sdb3 from 2nd disk (or 3rd, 4th, etc)
  3. For this post, I will use TrueNAS to create new Zvol called vg00/zvol0 (storage pool vg00, Zvol/dataset zvol0)
  4. Adds vg00/zvol0 into boot-pool
    1. Command: zpool add boot-pool vg00/zvol0
  5. Detach sda3 from boot-pool
    1. Command: zpool remove boot-pool sda3
    2. This command will take some time as it is going to copy 3.18GB into vg00/zvol0
  6. Re-create partition sda3 to smaller size, e.g. 100GB
    1. Command: parted sda
    2. Refers to parted command on how to delete partition using menu driven command
    3. Refers to parted command on how to create partition using menu driven command.  It must be larger than 3.18GB to hold all the data
  7. Adds sda3 back into boot-pool
    1. Command: zpool add boot-pool sda3
  8. Detach vg00/zvol0 from boot-pool
    1. Command: zpool remove boot-pool vg00/zvol0
    2. This command will take some time as it is going to copy 3.18GB into sda3


2025-08-01

OpenVPN: Windows RDC Remote Desktop Configuration

Product: OpenVPN, Windows OS
Software: MS Remote Desktop Services
Version: all

Sharing my experience of configuring OpenVPN that bundled with ASUS router which released in 2015.

  1. Login to ASUS WiFi router as admin
  2. Click on VPN button
  3. Click on tab "VPN Server"
  4. Enable OpenVPN server
    1. For my ASUS access point (I will use AP in later post), it allows me to configure 2 OpenVPN servers
    2. The "Server instance" for them are:
      1. Server 1
      2. Server 2
  5. Under list box "VPN Details", clicks on "Advance" in the list box to show more VPN server configurations
  6. Sets following 3 VPN server parameters
    1. Advertise DNS to clients = Yes
    2. Respond to DNS = Yes
    3. Optionally set Direct clients to redirect Internet traffic = Yes. This will allows other application think your traffic is originating from home computer instead of you current position. E.g. Facebook, YouTube, eBay, Amazon
  7. Following server parameters are optional to change to increase security, and minimize hacker
    1. Server port
    2. Firewall
    3. Username/Password authentication
    4. TLS control channel security
    5. Auth digest
    6. VPN subnet/netmask
  8. Clicks on "Apply" button to start the VPN server
  9. Now the OpenVPN server is completed
  10. Navigate back to the same screen, if it forward you to other page
  11. "VPN Details" will change back to "General" which will then shows button "Export" for you to download .opvn file
  12. The browser will prompt you to save the OpenVPN client config file with .opvn file extension
  13. Inside the browser's file download area, it will ask to reject or keep the file. For me, it doesn't prompt me anything, and I have to click on download button (in Chrome), then I can see it is showing an additional "Keep" button next to the .opvn file
    1. Clicks on "Keep" button next to the .opvn file to confirm file download
  14. Now, distribute the OpenVPN client configuration files to machines, and mobile devices where you want to use OpenVPN client. For examples
    1. Windows PC - copy the file to the copy through file sharing, or USB thumbdrives
    2. Apple iOS such as iPhone, iPad - uses e-mail file attachment to send to the email addresses where your iOS devices (iPhone, iPad) has e-mail configured.  If you have not setup e-mail, please set it up, and you can remove it after you have OpenVPN client setup
    3. Android - I recommend to use e-mail as well, unless you have SD card
  15. For each of the devices or PC you have, install OpenVPN client, which then you can use to import the .opvn file
  16. By now, OpenVPN has been running inside the ASUS AP, and OpenVPN client configuration files are readily to be used (and distributed)
  17. In Windows OS where you want to RDC into it, open "Windows Defender Firewall"
    1. For Windows 11, navigation nis Control Panel > System and Security > Windows Defender Firewall
  18. You should see following 3 network area:
    1. Private network - this is the firewall configuration for machines which has the identical IP address of this Windows OS. For example, if the Windows's IP is 192.168.1.1, then this is for incoming RDC request from 192.168.1.1 - 192.168.1.254
    2. Domain network - this is optional. If it shows up, then typically it is for local LAN but with different IP than 192.168.1.1
    3. Guest or public networks - starting Windows 11, both WiFi and LAN adapters are set to public network, and will block incoming RDC connection
    4. Follows this step if want to change the LAN or WiFi adapter to "Private" network for Windows 11
      1. Click on Start > Your name
      2. Next to your Avator icon, click on "My Microsoft account" to open configuration menu
      3. You should land on Home menu
      4. Look at the 4th menu, called "Network & internet"
      5. Clicks on menu "Network & internet"
      6. You can only change its profile if the network adapter is connected.  So if you want to change
        1. WiFi adapter, then connects your WiFi
        2. LAN adapter, then connects LAN cable
      7. Clicks on WiFi or LAN adapter
      8. Clicks on WiFi SSID which is the 2nd button below WiFi on/off button
      9. For example

      10. Clicks on "Private network" to change its profile to "Private network" which will allow incoming RDC connection
      11. If your computer belongs to a domain, then clicks on "Domain network" instead
      12. Now you are clear which network profile to pick in subsequent step of Windows Defender Firewall when choosing network profile of private, domain, public
      13. Runs "Windows Defender Firewall with Advanced Security"
        1. Clicks on "Inbound Rules"

        2. Scroll down to "Remote Desktop - User Mode (TCP-In)"
        3. Confirms the Profile = All, which means it allows incoming TCP connection to default RDC port 3389 from private, public, and domain profile
        4. Right clicks and select "Enable Rule"
      14. Now your computer will be able to accept incoming RDC connection from any IP in this network adapter/IP
      15. By now, following are done
        1. ASUS AP configured with OpenVPN server
        2. OpenVPN server is running
        3. OpenVPN client configuration .opvn file is distributed
        4. Windows machine configured to accept incoming RDC connection
      16. In each of the device you would like to run OpenVPN client, import the .opvn file
        1. Installs OpenVPN client software
        2. Windows - download from https://openvpn.net/client/
        3. iOS - search and download from Apple Store
        4. Android - search and download from Google Play
      17. Import the .opvn file
        1. Windows - double click on .opvn file
        2. iOS - open the e-mail which contains the .opvn attachment. Touch the .opvn attachment to show various options of opening the file. Choose "OpenVPN" icon
          1. OpenVPN app will run
          2. It will automatically trigger import profile screen
          3. Gives the profile your desire name, if you don't like the default cryptic name
        3. Android - open the e-mail which contains the .opvn attachment. Touch the .opvn attachment to show various options of opening the file. Choose "OpenVPN" icon
          1. OpenVPN app will run
          2. It will automatically trigger import profile screen
          3. Gives the profile your desire name, if you don't like the default cryptic name
      18. If you device and the Windows (RDC target host) are connecting to the same network, then in your OpenVPN client device, disconnect from the LAN/WiFi, and uses the Internet from your cellphone
      19. Turn on the OpenVPN to connect to ASUS AP against the newly imported profile
      20. Now, the device should be able to RDC to the target Windows using mstsc.exe command


2025-07-17

Linux: Dynamically/Realtime Changing ulimit for existing process

Product: RedHat Enterprise Linux RHEL
Version: All

Overview

Many UNIX/Linux software requesting to set process' limit using ulimit command.  This includes Oracle RDBMS, Apache httpd, Apache Tomcat, any file server, Cognos Analytics, call center servers, media server, Java base server, etc.

No vendor in any major application explain the objective of setting ulimit, and eventually the daemon/process hang or crash due to hitting such OS limit per user.

For reader who has no idea what is ulimit, the short description is that this is to configure user level's OS limit for any process run and own by the user.  This includes number of active open network connection, number of active open files, max file limit, max process RAM, etc.  This is not available in Windows, so anyone who never proper learn UNIX, they will miss this, or even miss this configuration.

Often, may IT personnel don't follow vendor's documentation to install the software.  Some vendor's document might even provided low value instead of a formula to properly tune the ulimit.  You need to understand that many developers do not understand OS, including UNIX OS, so it is challenging for a someone who don't understand OS to provide OS tuning parameter and value.

Eventually, this leads to many daemons hanging and even crash.  As an OS administrator, or software support personnel, it will be helpful to resolve the hanging, or even prevent it reach that critical state when the daemon start to behaving poorly, if you are able to catch it before it crash.

This post introduce a Linux commands which you can change the ulimit at in real time without restarting the daemon, or OS.  This command required to be run as root:
  • RedHat RHEL - prlimit
    • Documentation: https://man7.org/linux/man-pages/man1/prlimit.1.html
    • Documentation: https://linux.die.net/man/2/prlimit
  • Ubuntu - chpst
    • Documentation: https://manpages.ubuntu.com/manpages/lunar/man8/chpst.8.html
For the rest of the post, I will just refer to RHEL command of prlimit, as most company will use RHEL to run their software.

Usage

Command prlimit can be used to change the ulimit for a specific process in real time without shutting it down, or reboot OS.

You need to always specify the process ID, or called PID in "ps" command's output.  So the syntax will always to include "-p <PID" such as "prlimit -p <PID of Oracle>"

Following ulimit configuration can be configure in real time:
  • RAM related
    • max data size (RAM) - parameter -d
    • max resident set size RSS (RAM) - parameter -m
    • max stack size (RAM > stack) - parameter -s
  • Storage related
    • max file size (storage) - parameter -f
    • max number of open files - parameter -n
    • max number of file locks - parameter -x
  • Messaging related
    • max number of bytes in POSIX message queue - parameter -q
  • Process
    • max number of processes - parameter -u
Each OS resource usage has a different command to check their current utilization, so you can search in Internet to identify how much is the usage before tune up these values.  I might write a new blog post if this page is getting sufficient hit, such as more than 50,000 hit.

By adjusting the ulimit of the active process in real time, you can avoid unplan downtime during business hour, and schedule a proper maintenance window to tune the OS.

2025-07-09

SAP IPS Logs

Product: SAP Information Platform Server (IPS, CMS), SAP BusinessObject Data Services (BODS)
Version: 4.2.x - 4.3.x
OS: Windows, Linux

There is no documentation about the log files created by SAP CMS software that bundled with SAP BODS software.  This post is mainly covering the log files

Following are the log files created by SAP CMS software, and they mix together with CMS' Tomcat application server:

aps_ORDSUPWDS01.AdaptiveProcessingServer_gc.log
aps_ORDSUPWDS01.AdaptiveProcessingServer_ncs.trc
aps_ORDSUPWDS01.AdaptiveProcessingServer_trace.000001.glf
aps_ORDSUPWDS01.EIMAdaptiveProcessingServer_gc.log
aps_ORDSUPWDS01.EIMAdaptiveProcessingServer_ncs.trc
aps_ORDSUPWDS01.EIMAdaptiveProcessingServer_trace.000001.glf
cms_ORDSUPWDS01.CentralManagementServer_ncs.trc
cms_ORDSUPWDS01.CentralManagementServer_trace.000001.glf
DS.AdminService_2025079_7444820959_0.log
DS.AdminService_2025079_7444820959_0.log.lck
DS.JobLauncherService_2025079_7444817762_0.log
DS.JobLauncherService_2025079_7444817762_0.log.lck
DS.RFCService_2025079_7444811058_0.log
DS.RFCService_2025079_7444811058_0.log.lck
fileserver_ORDSUPWDS01.InputFileRepository_ncs.trc
fileserver_ORDSUPWDS01.OutputFileRepository_ncs.trc
ICC.MetadataService_ord-sup-wds01_2025079_74448304_60_0.log
ICC.MetadataService_ord-sup-wds01_2025079_74448304_60_0.log.lck
ICC.ViewdataService_ord-sup-wds01_2025079_74448177_63_0.log
ICC.ViewdataService_ord-sup-wds01_2025079_74448177_63_0.log.lck
jobserver_ORDSUPWDS01.AdaptiveJobServer_ncs.trc
jobserver_ORDSUPWDS01.AdaptiveJobServer_trace.000001.glf
ORDSUPWDS01_gc.log
SIA_ORDSUPWDS01_trace.000001.glf

Purpose
  1. File extension ".lck" is lock file or a flag file which to imply the CMS daemon is running.  CMS has following daemons, and each has their own .lck lock file
    1. DS.AdminService_2025079_7444820959_0.log.lck
    2. DS.JobLauncherService_2025079_7444817762_0.log.lck
    3. DS.RFCService_2025079_7444811058_0.log.lck
    4. ICC.MetadataService_ord-sup-wds01_2025079_74448304_60_0.log.lck
    5. ICC.ViewdataService_ord-sup-wds01_2025079_74448177_63_0.log.lck
  2. File suffix "_gc" and extention ".log" (...._gc.log) is Java garbage collection log.  CMS daemon has no reason to has high Java memory usage, so these log files can be ignore
  3. The rest of the log files are useful for troubleshoot SAP CMS daemons, and needs to read everyone of them

2025-06-06

SAP BODS Designer Central Repository Not Showing Up When Adding

Product: SAP Data Services
Version: 4.2.x - 4.3.x

Problem Description

There are multiple BODS central repositories in the system.  Few of the central repositories are cloned from existing central repositories for following reasons:

  1. Setting up more central repositories in identical BODS versions for current env
  2. Setup another central repositories in identical BODS versions for different env. Issue occurs after 2nd attempt of refresh after DS Designer added the central repositories on 1st attempt
  3. During upgrade, clone into a different DB user (Oracle), or logical database (MS SQL Server), then uses BODS Repository Manager to upgrade it without disrupting older BODS version
In DS Designer, menu Tools > Central Repository


After clicked "Add" button, it is either display a screen to add more central repository, or following message:

Even if it shows a list of central repositories to add, users are not able to see the desired central repository from the list.

DSMC central repository user and group setup has been done, even for secured central repository, when login to DS Designer as administrator user account.


Information to Gather

1. Login to central repository and determine the unique ID for the central repository.  This is called GUID, which will be added into local repository when users add it.  Run following SQL

select guid from al_version;

2. Repeat above for each of the central repository to get all the GUID

3. Login to local repository which failed to see other central repository

4. Run following SQL to show the central repository name and central repository GUID

select name, guid, object_key, object_type from al_lang where object_type = 5 and object_key in (select parent_objid from al_setoptions where parent_objid = al_lang.object_key and option_name = 'datastore_repotype' and option_value = 'central');

Sample output:

5. Explanation of above in local repository table AL_LANG

5.1. AL_LANG stores all object entries, including secure and non-secure central repositories

5.2. object_type = 5 is for datastores, secure repositories, non-secure repositories

5.3. al_SetOptions stores more detail for secure and non-secure central repository for Option_Name = datastore_repotype with Option_Value = 'central'

5.4. If you want to further filter by secure repository, then filter al_SetOptions by Option_Name = CENTRAL_REPO_SECURE, Option_Value = yes

Analysis

1. Compares the GUID from Step 1 with Step 4 for the same central repository name

2. Identify which central repository has the same GUID

3. This should be the central repository which is not visible in DS Designer's adding central repository screen

4. If you removed the central repository with the same GUID from DS Designer, then the list will be able to show all the central repository, even their GUID are identical

5. However, once you added one of those duplicate GUID into DS Designer, when try to add central repository again, you won't be able to see the central repository that has duplicate GUID

6. This is a consistent behavior in all local repositories

Root Cause

The AL_Version table was cloned from one central repository to another one, which leads to identical unique ID on column GUID.  Local repositories expect each central repository to have a unique GUID, else DS Designer assumes the central repository (with the same GUID) has been added, and will hide it.

Resolution

The supported approach is to use Repository Manager to re-initialize the central repository (all content will be lost), so that it will assign a new GUID.  Due to all central repository's content will be lost, if you want to keep its existing content, then find a local repository which can be used to check-out all content into it, then check back in to the central repository after it is initialized