Tuesday, February 11, 2025

PostgreSql: Useful Commands-

 1)   ************************* Users ***************************
 -- List of users with roles assigned:
SELECT usename AS role_name,
  CASE
     WHEN usesuper AND usecreatedb THEN
   CAST('superuser, create database' AS pg_catalog.text)
     WHEN usesuper THEN
    CAST('superuser' AS pg_catalog.text)
     WHEN usecreatedb THEN
    CAST('create database' AS pg_catalog.text)
     ELSE
    CAST('' AS pg_catalog.text)
  END role_attributes
FROM pg_catalog.pg_user
ORDER BY role_name desc;
select * from pg_roles where rolname='database_user';  ---- rolcanlogin : True/False
SELECT usename, datname, state FROM pg_stat_activity;
SELECT usename,datname,state,client_addr,xact_start,query_start, backend_start, state_change,query FROM pg_stat_activity
where usename not like 'rds%';
-----------------------------------------------
2) *************************** Table *******************************
---- Table size:
SELECT pg_size_pretty( pg_total_relation_size('tablename') );

SELECT count(*)  FROM pg_tables WHERE tableowner = 'stg_user' and schemaname like 'stg%';
SELECT schemaname,tablename,tableowner  FROM pg_tables WHERE tableowner = 'stg_user' and schemaname like 'stg%' order by tablename;
SELECT schemaname,count(*)  FROM pg_tables WHERE tableowner = 'stg_user' and schemaname like 'stg%' group by schemaname  ;


3) ****************************** Performance queries ******************

long running query:

SELECT
  pid,
  now() - pg_stat_activity.query_start AS duration,
  query,
  state
FROM pg_stat_activity
WHERE (now() - pg_stat_activity.query_start) > interval '5 minutes';

another query with Idle session.

It is. And no, idle queries aren’t considered LRQ at all. When state is idle, the query shows information for the last executed query.
 For the purpose of the article -identify LRQ- the query should be modified to the following:

SELECT
  pid,
  user,
  pg_stat_activity.query_start,
  now() - pg_stat_activity.query_start AS query_time,
  query,
  state,
  wait_event_type,
  wait_event
FROM pg_stat_activity
WHERE state != 'idle' and (now() - pg_stat_activity.query_start) > interval '5 minu


How to check which query is taking long time in PostgreSQL?

Using log_min_duration_statement parameter: To use the log_min_duration_statement parameter to trace slow queries in PostgreSQL. 
If you configure log_min_duration_statement in postgresql. conf as 3500,
PostgreSQL will categorize queries lasting more than 3.5 seconds as slow queries and record them in the log file.

4 ********************** Kill/Delete/blocking/locks sessions *************************

In order to cancel these long running queries you should execute:

SELECT pg_cancel_backend(__pid__);

It may take a few seconds to stop the query entirely using the pg_cancel_backend command.

If the you find the process is stuck you can kill it by running:

SELECT pg_terminate_backend(__pid__);


pg_locks: Information on current locks held within the database by open transactions, with one row per lockable object.
pg_blocking_pids(): A function that can find the process IDs (PIDs) of sessions that are blocking the PostgreSQL server process of a supplied PID.
pg_cancel_backend(): Function that cancels the currently running query by sending a SIGINT to a process ID.
pg_terminate_backend(): Terminate a backend process completely (the query and usually the connection) on the database (uses SIGTERM instead of SIGINT).
-------
show the blocked processes on your database along with the actual query that is blocking them is:
SELECT
    activity.pid,
    activity.usename,
    activity.query,
    blocking.pid AS blocking_id,
    blocking.query AS blocking_query
FROM pg_stat_activity AS activity
JOIN pg_stat_activity AS blocking ON blocking.pid = ANY(pg_blocking_pids(activity.pid));

granted: True if lock is held, false if the process is waiting.
locktype: The type of object that is locked – (see here) relation, page, object, tranasctionid, userlock etc.

------

Joining the pg_lock view to pg_stat_activity for the query and pg_class for the table names can be useful to gather more context as to what is locking on your database at any point in time

select 
    relname as relation_name, 
    query, 
    pg_locks.* 
from pg_locks
join pg_class on pg_locks.relation = pg_class.oid
join pg_stat_activity on pg_locks.pid = pg_stat_activity.pid;


5************************ Cron Jobs *******************************

Pg cron: setup and
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_pg_cron.html

show all; --- to see all parameter values.

select * from cron.job_run_details order by start_time desc;

select * from cron.job;

SELECT name, setting, short_desc FROM pg_settings WHERE name LIKE 'cron.%' ORDER BY name;




*****************************************************************************************************************************************************************************

Monday, February 10, 2025

Random- finding users in DB and validate in AD

 first generate the list of DB users to the file for 10.11 and save to a text file in a location


open CMD

cd to the save txt file location and run below command.


for /f %i in (user_list.txt) do (echo %i & net user /domain %i |findstr Full) >>"user_log_output_2.txt"


for /f %i in (list-users.txt)

do 

command=

(echo %i & net user /domain %i |findstr Full) >>"user_log_output_2.txt"

Mysql: Handy commands when looking for any issues (mysqlbinlog)/diag report/mysql setting.

 

1) 

mysqlbinlog commands:

--- to start from specific time:

mysqlbinlog --start-datetime="2023-07-28 12:00:00" --base64-output=AUTO --verbose ctdayasd001-bin.000005 >/home/oracle/siva/asd_bin005.log


mysqlbinlog -d database_name --base64-output=AUTO --verbose mysqld-bin.000001 >/home/oracle/siva/test.log


--debug-check option will check for open file and memory use after it finish processing the given binary log file

mysqlbinlog -d database_name --debug-check mysqld-bin.000001


--debug-info option will display additional debugging option after it finish processing the given binary log file

mysqlbinlog -d database_name --debug-info mysqld-bin.000001

~


2) to run diag collect: like AWR for current time.

runs for 300 seconds(5 mins) for every 30 seconds to collect the details and report it.

mysql -u root -p -H -e"CALL sys.diagnostics(300, 30, 'current');" > /opt/mysql/current_Host_instance_report.html



3)

--- HTML report to check the mysql settings:

vi collect-mysql-det.sql

SELECT VERSION(), NOW(), @@time_zone, @@system_time_zone\G

/*!50503 SELECT ENGINE, COUNT(*), SUM(DATA_LENGTH) AS DATA, SUM(INDEX_LENGTH) AS IDXS, ((SUM(DATA_LENGTH)+SUM(INDEX_LENGTH))/(1024*1024)) AS TOTALMB

FROM information_schema.TABLES GROUP BY ENGINE*/;

select benchmark(50000000,(1234*5678/37485-1298+8596^2)); #should take less than 20 seconds

SHOW GLOBAL VARIABLES;

SELECT * FROM INFORMATION_SCHEMA.PLUGINS;

/*!80001 SELECT * FROM mysql.component*/;

-- part of an IC?

/*!50706 SELECT * FROM `performance_schema`.`replication_group_members`*/;

/*!50604 select processlist_state,count(*) from performance_schema.threads group by processlist_state*/;

SHOW GLOBAL STATUS;

SELECT SLEEP(180);

/*!50604 select processlist_state,count(*) from performance_schema.threads group by processlist_state*/;

SHOW GLOBAL STATUS;

STATUS;


MySQL MEB FULL Backup and Restore Testing (incremental)

 



MySQL MEB FULL Backup and Restore Testing


Here for this, we are testing the scenarios on Test server (TestServer). The backup and restore is little different from Oracle point of view. So we need to make sure we are testing MySQL as is with available functionality.  Do follow below steps to test the scenario to make sure we can put “Incremental Backups” in line.

1. Restore the database from Prod backup on to Test Server (TestServer). Once restored, Make sure can login into MySQL database and query tables and rows. Edit the shell with backup location.


Use this script: /opt/mysql/mysql-meb-restore_auto.sh


2. Once confirmed, take MEB backup for Test Server (TestServer).

Use this script: 

/opt/mysql/scripts/mysql-meb-backup.sh FULL 14 


3. Once backed up, make sure you can login into MySQL database and query tables and rows and truncate below tables.

select table_schema,table_name,table_rows from information_schema.tables where

table_schema='scott' and table_rows < 20000 order by 3 desc;


select count(*) from scott.Table1;

select count(*) from scott.Table2;

select count(*) from scott.Table3;

select count(*) from scott.Table4;


example: 

truncate scott.Table1;


4. We now take the incremental backup on the Test Server (TestServer).

Use this script: 

/opt/mysql/scripts/ mysql-meb-incr-backup.sh INCR 14


5. Now, we try to restore MySQL as full backup and test it. edit the shell with backup location.

Use this script: 

/opt/mysql/ mysql-meb-restore_auto_host.sh


6. Once restored, login and see the table rows of the above truncated tables. You should see all rows.


7. Now, do the incremental restore to get to the current state. Edit the shell with backup location.

Use script:

/opt/mysql/mysql-meb-restore_auto_incremental.sh


8. Once restore, login to MySQL database and check the truncate table row counts.


Point-in-time recovery:

Point-in-time is manual process in MySQL not like that we do in Oracle.

MySQL doesn’t recommend using timestamp to do point-in-time recovery. Instead we need to restore the database to a logfile before the event logfile happened and then apply the changes of the event logfile using start lsn and end lsn. Below link you can take a look for Point-in-time recovery.

Using the --start-datetime or --stop-datetime option to specify the range of binary log segment to replay is not recommended: there is a higher risk of missing binary log events when using the option. Use --start-position and --stop-position instead.



Links for More Information: 

https://dev.mysql.com/doc/mysql-enterprise-backup/4.1/en/mysqlbackup.incremental.html

https://dev.mysql.com/doc/mysql-enterprise-backup/8.4/en/backup-incremental-options.html

https://dev.mysql.com/doc/mysql-enterprise-backup/4.1/en/restore.incremental.html

https://dev.mysql.com/doc/refman/8.4/en/analyze-table.html

https://dev.mysql.com/doc/mysql-enterprise-backup/4.1/en/advanced.point.html




Tuesday, August 20, 2024

PL/sql code to delete rows from table.

 -- DDL TO CREATE DEBUGGING TABLE -----> Monitor First time

-- create TABLE TRACK_TAB(ROWS_DELETED NUMBER,TOTAL NUMBER,START_TIME TIMESTAMP,END_TIME TIMESTAMP,DESC VARCHAR2(60))

-- create index TEST_TAB_INDX on TEST_TAB( CREATION_DATE)

DECLARE

nCount NUMBER:=0;

iCount NUMBER:=0;

 batchSize NUMBER:=5000;

daysOld NUMBER:=30;


 sql1 VARCHAR2(2000);

 starttime timestamp;

BEGIN

sql1 := 'DELETE FROM TEST_TAB WHERE CREATION_DATE < sysdate - ' || daysOld ||' and rownum <= ' || batchSize;

LOOP

starttime:=sysdate;

EXECUTE IMMEDIATE sql1;

nCount := sql%rowcount;

iCount:=iCount+nCount;

insert into TRACK_TAB values (batchSize,iCount,starttime, sysdate,'DELETE old TEST_TAB records ');


commit;

 EXIT WHEN nCount = 0;


END LOOP;

end;

Friday, April 28, 2023

Mysql: find tables with no primary key

 SELECT t.table_schema,
       t.table_name,
       k.constraint_name,
       k.column_name
FROM information_schema.tables t
     LEFT JOIN information_schema.key_column_usage k
          ON t.table_schema = k.table_schema
             AND t.table_name = k.table_name
             AND k.constraint_name = 'PRIMARY'
 WHERE t.table_schema NOT IN ( 'mysql', 'information_schema',
                                       'performance_schema' )
       AND k.constraint_name IS NULL
       AND t.table_type = 'BASE TABLE'
       order by table_schema desc;

Wednesday, April 19, 2023

how to skip commented lines in linux in loop for shell script.

How to skip commented input lines from a input file in loop in Linux script:
 
below is the simple code with small example:


#!/bin/ksh
# Script to check required ORACLE - RHEL RPMs:
#
while read rpm_c
  do
case "$rpm_c" in \#*) continue ;; esac
rpm_check=`rpm -qa $rpm_c | wc -l`
if [[ $rpm_check != 0 ]]; then
rpm_check=YES
echo $'\e[1;36m' rpm: $'\e[1;34m' $rpm_c   $'\e[1;32m' $rpm_check $'\e[0m'
else
rpm_check=NO
echo $'\e[1;36m' rpm: $'\e[1;34m' $rpm_c  $'\e[1;31m'  $rpm_check $'\e[0m'
fi
done<RHEL-7_ORACLE19C_RPMS.log


input in the logfile:


libgcc
libstdc++
libstdc++-devel
libxcb
make
smartmontools
sysstat
#
#Optional Packages for Red Hat Enterprise Linux 7:
#
#ipmiutil (for Intelligent Platform Management Interface)
#net-tools (for Oracle RAC and Oracle Clusterware)
#nfs-utils (for Oracle ACFS)
#python (for Oracle ACFS Remote)

PostgreSql: Useful Commands-

 1)   ************************* Users ***************************  -- List of users with roles assigned: SELECT usename AS role_name,   CASE...