主机分配

主机名       IP                        配置    用途
shardgsm 172.17.201.101   2U3G 分片目录
shardcatdb 172.17.201.102 4U5G 配置库
shard1 172.17.201.103       4U5G 分片数据库1
shard2 172.17.201.104       4U5G 分片数据库2

/etc/hosts
172.17.201.101 shardgsm shardgsm.localdomain
172.17.201.102 shardcat shardcat.localdomain
172.17.201.103 shard1 shard1.localdomain
172.17.201.104 shard2 shard2.localdomain

systemctl stop firewalld
systemctl disable firewalld

GSM安装

# xhost +
# su - oracle
Last login: Sat Dec 9 16:08:35 CST 2017 on pts/0
$ export DISPLAY=172.17.201.10:0.0
$ ./runInstaller

/u01/app/oraInventory/orainstRoot.sh
/u01/app/oracle/product/12.2.0/db_1/root.sh

RDBMS安装
在另外三个节点进行安装

xhost +
su - oracle
export DISPLAY=172.17.201.10:0.0

cd /u01/soft/
unzip linuxx64_12201_database.zip

$ ./runInstaller

/u01/app/oraInventory/orainstRoot.sh
/u01/app/oracle/product/12.2.0/db_1/root.sh

配置库

dbca创建配置库

在shardcat节点安装数据库
mkdir -p /u01/data/arch
mkdir -p /u01/data/fra
chown -R oracle.oinstall /u01/data
非容器，不安装OEM，开启归档、闪回
dbca -silent -deleteDatabase -sourceDB catdb
dbca -silent -createDatabase -templateName General_Purpose.dbc -gdbname catdb -sid catdb \
-sysPassword Admin_123 -systemPassword Admin_123 -responseFile NO_VALUE \
-storageType FS -datafileDestination /u01/data -redoLogFileSize 500 \
-enableArchive true -archiveLogDest /u01/data/arch \
-recoveryAreaDestination /u01/data/fra -recoveryAreaSize 20480 \
-characterSet AL32UTF8 -nationalCharacterSet AL16UTF16 -sampleSchema true -memoryPercentage 30 -databaseType OLTP -emConfiguration NONE

配置监听

[oracle@shardcat admin]$ cd /u01/app/oracle/product/12.2.0/db_1/network/admin
[oracle@shardcat admin]$ cp samples/listener.ora .
[oracle@shardcat admin]$ vim listener.ora
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = shardcat)(PORT = 1521))
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
)
)
[oracle@shardcat admin]$ lsnrctl start
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=shardcat)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=shardcat)(PORT=8080))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "catdb" has 1 instance(s).
Instance "catdb", status READY, has 1 handler(s) for this service...
Service "catdbXDB" has 1 instance(s).
Instance "catdb", status READY, has 1 handler(s) for this service...
The command completed successfully
[oracle@shardcat ~]$

账户配置

alter user gsmcatuser identified by Admin_123 account unlock;
ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

gsm帐户是数据库管理员模式，用于对sharded数据库环境进行管理更改。当运行GDSCTL命令时，GDSCTL通过该用户连接到数据库，gsm用户在数据库中进行必要的更改。

create user gsm identified by Admin123 default tablespace users quota unlimited on users;
grant gsmadmin_role to gsm;
grant connect,create session to gsm;
grant inherit privileges on user SYS to GSMADMIN_INTERNAL;

execute dbms_xdb.sethttpport(8080);
commit;
@?/rdbms/admin/prvtrsch.plb
exec DBMS_SCHEDULER.SET_AGENT_REGISTRATION_PASS('Admin_123');

参数配置

alter system set db_create_file_dest='/u01/data/catdb' scope=both;
alter system set open_links=16 scope=spfile;
alter system set open_links_per_instance=16 scope=spfile;
create pfile from spfile;

重启数据库

为了使配置的参数生效，重启数据库实例
shutdown immediate

GSM管理

登录GSM服务器

create配置库

[oracle@shardgsm admin]$ cat tnsnames.ora
catdbtns =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = shardcat.localdomain)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = catdb)
)
)

[oracle@shardgsm gsm]$ gdsctl
GDSCTL: Version 12.2.0.1.0 - Production on Sat Dec 09 20:01:22 CST 2017
Current GSM is set to GSMORA
GDSCTL>
create shardcatalog -database catdbtns -chunks 36 -user gsm/Admin123 -sdb sdbname -region region1, region2 -agent_port 8080 -agent_password Admin_123

-sdb sharding库的名字，可自定义
-region 创建两个区域
-agent_port配置库服务器上开放的供shard库注册的端口，默认8080

shard director
connect gsm/Admin123@catdbtns
add gsm -gsm gsmdrt1 -listener 1526 -pwd Admin_123 -catalog catdbtns -region region1
start gsm -gsm gsmdrt1

GDSCTL>status gsm
Alias GSMDRT1
Version 12.2.0.1.0
Start Date 09-DEC-2017 20:53:34
Trace Level off
Listener Log File /u01/app/oracle/diag/gsm/shardgsm/gsmdrt1/alert/log.xml
Listener Trace File /u01/app/oracle/diag/gsm/shardgsm/gsmdrt1/trace/ora_9573_140348623991168.trc
Endpoint summary (ADDRESS=(HOST=shardgsm)(PORT=1526)(PROTOCOL=tcp))
GSMOCI Version 2.2.1
Mastership Y
Connected to GDS catalog Y
Process Id 9575
Number of reconnections 0
Pending tasks. Total 0
Tasks in process. Total 0
Regional Mastership TRUE
Total messages published 0
Time Zone +08:00
Orphaned Buddy Regions:
None
GDS region region1

添加操作系统认证
用于执行dbca等操作的用户。
GDSCTL>add credential -credential credos -osaccount oracle -ospassword oracle

注册shard到配置库
登录各个shard分片服务器
创建必要目录
mkdir /u01/app/oracle/oradata
mkdir /u01/app/oracle/fast_recovery_area
chown -R oracle.oinstall /u01/app/oracle/oradata
chown -R oracle.oinstall /u01/app/oracle/fast_recovery_area

启动agent，并注册数据库
schagent -start
echo Admin_123 | schagent -registerdatabase shardcat 8080

shardcat：配置库主机名
8080：配置库开放的端口

注意：如果注册失败，并导致用户锁住，要重新执行
SQL> @?/rdbms/admin/prvtrsch.plb
SQL> exec DBMS_SCHEDULER.SET_AGENT_REGISTRATION_PASS('Admin_123');

部署System-Managed SDB

先决条件

They must not be container databases (CDBs)
They must have an associated TNS Listener on port 1521 on each host
The GSMUSER account must be unlocked with a known password
The primary and standby databases must be configured as such
Redo apply should be set up between the corresponding primary and standby databases
Flashback and force logging should be enabled
The compatible parameter must be set to at least 12.2.0
A server parameter file (SPFILE) must be in use
A DATA_PUMP_DIR directory object must be created in each database and must point to a valid directory

shard库检查

SQL> set serveroutput on
SQL> execute DBMS_GSM_FIX.validateShard
INFO: Data Guard shard validation requested.
INFO: Database role is PRIMARY.
INFO: Database name is CATDB.
INFO: Database unique name is catdb.
INFO: Database ID is 2507992168.
INFO: Database open mode is READ WRITE.
INFO: Database in archivelog mode.
WARNING: Flashback is off.
ERROR: Force logging is off.
INFO: Database platform is Linux x86 64-bit.
INFO: Database character set is AL32UTF8. This value must match the character
set of the catalog database.
INFO: 'compatible' initialization parameter validated successfully.
INFO: Database is not a multitenant container database.
INFO: Database is using a server parameter file (spfile).
INFO: db_create_file_dest set to: '/u01/data/catdb'
INFO: db_recovery_file_dest set to: '/u01/data/fra'
INFO: db_files=200. Must be greater than the number of chunks and/or tablespaces
to be created in the shard.
ERROR: dg_broker_start set to FALSE.
INFO: remote_login_passwordfile set to EXCLUSIVE.
WARNING: db_file_name_convert is not set.
ERROR: GSMUSER account status is invalid: EXPIRED & LOCKED
ERROR: GSMADMIN_INTERNAL does not have READ privilege on DATA_PUMP_DIR.
ERROR: GSMADMIN_INTERNAL does not have WRITE privilege on DATA_PUMP_DIR.
INFO: DATA_PUMP_DIR is '/u01/app/oracle/admin/catdb/dpdump/'.

PL/SQL procedure successfully completed.

开启闪回、强制日志、DG_BROKER
SQL> shutdown immediate
SQL> startup mount;
SQL> alter database flashback on;
SQL> alter database force logging;
SQL> alter system set dg_broker_start=true;
SQL> show parameter dg_broker

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
dg_broker_config_file1 string /u01/app/oracle/product/12.2.0
/db_1/dbs/dr1catdb.dat
dg_broker_config_file2 string /u01/app/oracle/product/12.2.0
/db_1/dbs/dr2catdb.dat
dg_broker_start boolean TRUE

SQL> alter database open;

解锁账户
SQL> alter USER GSMUSER IDENTIFIED BY Admin_123 account unlock;

dump目录授权
[root@shardcat ~]# mkdir /u01/dumps
[root@shardcat ~]# chown -R oracle.dba /u01/dumps/
CREATE OR REPLACE DIRECTORY DATA_PUMP_DIR as '/u01/dumps';
grant read,write on directory DATA_PUMP_DIR to GSMADMIN_INTERNAL;

连接配置库

[oracle@shardgsm ~]$ gdsctl
Current GSM is set to GSMDRT1
GDSCTL>connect gsm/Admin123@catdbtns
Catalog connection is established

添加shardgroup

GDSCTL>add shardgroup -shardgroup primary_shardgroup -deploy_as primary -region region1
Catalog connection is established
The operation completed successfully

添加shard database节点并创建shards

GDSCTL>add invitednode shard1
GDSCTL>create shard -shardgroup primary_shardgroup -destination shard1 -credential credos -sys_password Admin_123
The operation completed successfully
DB Unique Name: sh1
GDSCTL>add invitednode shard2
GDSCTL>create shard -shardgroup primary_shardgroup -destination shard2 -credential credos -sys_password Admin_123
The operation completed successfully
DB Unique Name: sh2

检查shard 配置

GDSCTL>config shard
Name Shard Group Status State Region Availability
---- ----------- ------ ----- ------ ------------
sh1 primary_shardgroup U none region1 -
sh2 primary_shardgroup U none region1 -

使用DEPLOY创建shards和复制数据
大约需要30分钟，需要在shards执行dbca。
GDSCTL>deploy
deploy: examining configuration...
deploy: deploying primary shard 'sh1' ...
deploy: network listener configuration successful at destination 'shard1'
deploy: starting DBCA at destination 'shard1' to create primary shard 'sh1' ...
deploy: deploying primary shard 'sh2' ...
deploy: network listener configuration successful at destination 'shard2'
deploy: starting DBCA at destination 'shard2' to create primary shard 'sh2' ...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: waiting for 2 DBCA primary creation job(s) to complete...
deploy: DBCA primary creation job succeeded at destination 'shard1' for shard 'sh1'
deploy: DBCA primary creation job succeeded at destination 'shard2' for shard 'sh2'
deploy: requesting Data Guard configuration on shards via GSM
deploy: shards configured successfully
The operation completed successfully

验证shards
GDSCTL>config shard
Name Shard Group Status State Region Availability
---- ----------- ------ ----- ------ ------------
sh1 primary_shardgroup Ok Deployed region1 ONLINE
sh2 primary_shardgroup Ok Deployed region1 ONLINE

[oracle@shard1 sh1]$ ll
total 2013452
-rw-r-----. 1 oracle oinstall 10600448 Dec 10 13:18 control01.ctl
-rw-r-----. 1 oracle oinstall 10600448 Dec 10 13:18 control02.ctl
-rw-r-----. 1 oracle oinstall 209715712 Dec 10 13:18 redo01.log
-rw-r-----. 1 oracle oinstall 209715712 Dec 10 12:52 redo02.log
-rw-r-----. 1 oracle oinstall 209715712 Dec 10 12:52 redo03.log
-rw-r-----. 1 oracle oinstall 492838912 Dec 10 13:17 sysaux01.dbf
-rw-r-----. 1 oracle oinstall 838868992 Dec 10 13:12 system01.dbf
-rw-r-----. 1 oracle oinstall 33562624 Dec 10 12:44 temp01.dbf
-rw-r-----. 1 oracle oinstall 73408512 Dec 10 13:12 undotbs01.dbf
-rw-r-----. 1 oracle oinstall 5251072 Dec 10 12:52 users01.dbf

[oracle@shard1 sh1]$ export ORACLE_SID=sh1
[oracle@shard1 sh1]$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Sun Dec 10 13:18:48 2017

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> show parameter arch

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
archive_lag_target integer 0
log_archive_config string
log_archive_dest string
log_archive_dest_1 string location=use_db_recovery_file_
dest mandatory valid_for=(all_logfiles,all_roles)

SQL> select name,force_logging,flashback_on from v$database;

NAME FORCE_LOGGING FLASHBACK_ON
--------- --------------------------------------- ------------------
SH1 YES YES

创建用户、表空间、表

为System-Managed SDB创建用户
alter session enable shard ddl;
create user sharduser identified by sharduser;
grant all privileges to sharduser ;
grant gsmadmin_role to sharduser ;
grant select_catalog_role to sharduser ;
grant connect, resource to sharduser ;
grant dba to sharduser ;
grant execute on dbms_crypto to sharduser ;

创建表空间
创建表空间集用存储shared表
CREATE TABLESPACE SET TSP_SET_1 using template (datafile size 100m autoextend on next 10M maxsize unlimited extent management local segment space management auto);

为Duplicated表创建表空间
CREATE TABLESPACE products_tsp datafile size 100m autoextend on next 10M maxsize unlimited extent management local uniform size 1m;

shard用户创建Sharded表和Duplicated表

[oracle@shardcat admin]$ sqlplus sharduser/sharduser
ALTER SESSION ENABLE SHARD DDL;

CREATE SHARDED TABLE Customers
(
CustId VARCHAR2(60) NOT NULL,
FirstName VARCHAR2(60),
LastName VARCHAR2(60),
Class VARCHAR2(10),
Geo VARCHAR2(8),
CustProfile VARCHAR2(4000),
Passwd RAW(60),
CONSTRAINT pk_customers PRIMARY KEY (CustId),
CONSTRAINT json_customers CHECK (CustProfile IS JSON)
) TABLESPACE SET TSP_SET_1
PARTITION BY CONSISTENT HASH (CustId) PARTITIONS AUTO;

CREATE SHARDED TABLE Orders
(
OrderId INTEGER NOT NULL,
CustId VARCHAR2(60) NOT NULL,
OrderDate TIMESTAMP NOT NULL,
SumTotal NUMBER(19,4),
Status CHAR(4),
CONSTRAINT pk_orders PRIMARY KEY (CustId, OrderId),
CONSTRAINT fk_orders_parent FOREIGN KEY (CustId)
REFERENCES Customers ON DELETE CASCADE
) PARTITION BY REFERENCE (fk_orders_parent);

CREATE SEQUENCE Orders_Seq;

CREATE SHARDED TABLE LineItems
(
OrderId INTEGER NOT NULL,
CustId VARCHAR2(60) NOT NULL,
ProductId INTEGER NOT NULL,
Price NUMBER(19,4),
Qty NUMBER,
CONSTRAINT pk_items PRIMARY KEY (CustId, OrderId, ProductId),
CONSTRAINT fk_items_parent FOREIGN KEY (CustId, OrderId)
REFERENCES Orders ON DELETE CASCADE
) PARTITION BY REFERENCE (fk_items_parent);

CREATE DUPLICATED TABLE Products
(
ProductId INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
Name VARCHAR2(128),
DescrUri VARCHAR2(128),
LastPrice NUMBER(19,4)
) TABLESPACE products_tsp;

验证上面的DDL操作是否有失败的
可以看到Failed shards是空的，表示所有操作均执行成功。
GDSCTL>show ddl
id DDL Text Failed shards
-- -------- -------------
5 grant connect, resource to sharduser
6 grant dba to sharduser
7 grant execute on dbms_crypto to shard...
8 CREATE TABLESPACE SET TSP_SET_1 using...
9 CREATE TABLESPACE products_tsp datafi...
10 CREATE SHARDED TABLE Customers ( ...
11 CREATE SHARDED TABLE Orders ( O...
12 CREATE SEQUENCE Orders_Seq
13 CREATE SHARDED TABLE LineItems ( ...
14 CREATE MATERIALIZED VIEW "SHARDUSER"....

配置库验证表空间集，表
配置库
select TABLESPACE_NAME, BYTES/1024/1024 MB from sys.dba_data_files order by tablespace_name;

sharduser登录
select table_name,
tablespace_name,
pct_free,
pct_used,
ini_trans,
max_trans,
logging,
partitioned,
has_identity
from user_tables;

select t.table_name,
t.partitioning_type,
t.partition_count,
t.partitioning_key_count,
t.def_tablespace_name,
t.ref_ptn_constraint_name,
t.def_pct_free,
t.def_pct_used
from USER_PART_TABLES t;

select table_name, partition_name, tablespace_name
from dba_tab_partitions
where tablespace_name like '%TSP_SET_1'
order by tablespace_name;

分片库1
[oracle@shard1 ~]$ sqlplus sharduser/sharduser
SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
CUSTOMERS
ORDERS
LINEITEMS
PRODUCTS

分片库2
[oracle@shard2 ~]$ sqlplus sharduser/sharduser
SQL> select table_name from user_tables;

TABLE_NAME
--------------------------------------------------------------------------------
CUSTOMERS
ORDERS
LINEITEMS
PRODUCTS

分片库验证表分片
[oracle@shard1 ~]$ sqlplus / as sysdba
SQL> show parameter db_unique_name

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_unique_name string sh1

set linesize 140
column table_name format a20
column tablespace_name format a20
column partition_name format a20
select table_name, partition_name, tablespace_name
from dba_tab_partitions
where tablespace_name like '%TSP_SET_1'
order by tablespace_name;

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
1 CUSTOMERS CUSTOMERS_P4 C004TSP_SET_1
106 CUSTOMERS CUSTOMERS_P2 C002TSP_SET_1
7 CUSTOMERS CUSTOMERS_P19 TSP_SET_1
8 CUSTOMERS CUSTOMERS_P20 TSP_SET_1
9 CUSTOMERS CUSTOMERS_P21 TSP_SET_1
10 CUSTOMERS CUSTOMERS_P22 TSP_SET_1
11 CUSTOMERS CUSTOMERS_P23 TSP_SET_1
12 CUSTOMERS CUSTOMERS_P24 TSP_SET_1
13 CUSTOMERS CUSTOMERS_P25 TSP_SET_1
14 CUSTOMERS CUSTOMERS_P26 TSP_SET_1
15 CUSTOMERS CUSTOMERS_P27 TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
16 CUSTOMERS CUSTOMERS_P28 TSP_SET_1
17 CUSTOMERS CUSTOMERS_P29 TSP_SET_1
18 CUSTOMERS CUSTOMERS_P30 TSP_SET_1
19 CUSTOMERS CUSTOMERS_P31 TSP_SET_1
20 CUSTOMERS CUSTOMERS_P32 TSP_SET_1
21 CUSTOMERS CUSTOMERS_P33 TSP_SET_1
22 CUSTOMERS CUSTOMERS_P34 TSP_SET_1
23 CUSTOMERS CUSTOMERS_P35 TSP_SET_1
24 CUSTOMERS CUSTOMERS_P36 TSP_SET_1
61 CUSTOMERS CUSTOMERS_P7 C007TSP_SET_1
64 CUSTOMERS CUSTOMERS_P9 C009TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
67 CUSTOMERS CUSTOMERS_P17 C00HTSP_SET_1
70 CUSTOMERS CUSTOMERS_P12 C00CTSP_SET_1
73 CUSTOMERS CUSTOMERS_P14 C00ETSP_SET_1
76 CUSTOMERS CUSTOMERS_P13 C00DTSP_SET_1
79 CUSTOMERS CUSTOMERS_P8 C008TSP_SET_1
82 CUSTOMERS CUSTOMERS_P10 C00ATSP_SET_1
85 CUSTOMERS CUSTOMERS_P11 C00BTSP_SET_1
88 CUSTOMERS CUSTOMERS_P16 C00GTSP_SET_1
91 CUSTOMERS CUSTOMERS_P6 C006TSP_SET_1
94 CUSTOMERS CUSTOMERS_P1 C001TSP_SET_1
97 CUSTOMERS CUSTOMERS_P15 C00FTSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
100 CUSTOMERS CUSTOMERS_P3 C003TSP_SET_1
103 CUSTOMERS CUSTOMERS_P5 C005TSP_SET_1
4 CUSTOMERS CUSTOMERS_P18 C00ITSP_SET_1
2 LINEITEMS CUSTOMERS_P4 C004TSP_SET_1
107 LINEITEMS CUSTOMERS_P2 C002TSP_SET_1
25 LINEITEMS CUSTOMERS_P19 TSP_SET_1
26 LINEITEMS CUSTOMERS_P20 TSP_SET_1
27 LINEITEMS CUSTOMERS_P21 TSP_SET_1
28 LINEITEMS CUSTOMERS_P22 TSP_SET_1
29 LINEITEMS CUSTOMERS_P23 TSP_SET_1
30 LINEITEMS CUSTOMERS_P24 TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
31 LINEITEMS CUSTOMERS_P25 TSP_SET_1
32 LINEITEMS CUSTOMERS_P26 TSP_SET_1
33 LINEITEMS CUSTOMERS_P27 TSP_SET_1
34 LINEITEMS CUSTOMERS_P28 TSP_SET_1
35 LINEITEMS CUSTOMERS_P29 TSP_SET_1
36 LINEITEMS CUSTOMERS_P30 TSP_SET_1
37 LINEITEMS CUSTOMERS_P31 TSP_SET_1
38 LINEITEMS CUSTOMERS_P32 TSP_SET_1
39 LINEITEMS CUSTOMERS_P33 TSP_SET_1
40 LINEITEMS CUSTOMERS_P34 TSP_SET_1
41 LINEITEMS CUSTOMERS_P35 TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
42 LINEITEMS CUSTOMERS_P36 TSP_SET_1
62 LINEITEMS CUSTOMERS_P7 C007TSP_SET_1
65 LINEITEMS CUSTOMERS_P9 C009TSP_SET_1
68 LINEITEMS CUSTOMERS_P17 C00HTSP_SET_1
71 LINEITEMS CUSTOMERS_P12 C00CTSP_SET_1
74 LINEITEMS CUSTOMERS_P14 C00ETSP_SET_1
77 LINEITEMS CUSTOMERS_P13 C00DTSP_SET_1
80 LINEITEMS CUSTOMERS_P8 C008TSP_SET_1
83 LINEITEMS CUSTOMERS_P10 C00ATSP_SET_1
86 LINEITEMS CUSTOMERS_P11 C00BTSP_SET_1
89 LINEITEMS CUSTOMERS_P16 C00GTSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
92 LINEITEMS CUSTOMERS_P6 C006TSP_SET_1
95 LINEITEMS CUSTOMERS_P1 C001TSP_SET_1
98 LINEITEMS CUSTOMERS_P15 C00FTSP_SET_1
101 LINEITEMS CUSTOMERS_P3 C003TSP_SET_1
104 LINEITEMS CUSTOMERS_P5 C005TSP_SET_1
5 LINEITEMS CUSTOMERS_P18 C00ITSP_SET_1
3 ORDERS CUSTOMERS_P4 C004TSP_SET_1
6 ORDERS CUSTOMERS_P18 C00ITSP_SET_1
43 ORDERS CUSTOMERS_P19 TSP_SET_1
44 ORDERS CUSTOMERS_P20 TSP_SET_1
45 ORDERS CUSTOMERS_P21 TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
46 ORDERS CUSTOMERS_P22 TSP_SET_1
47 ORDERS CUSTOMERS_P23 TSP_SET_1
48 ORDERS CUSTOMERS_P24 TSP_SET_1
49 ORDERS CUSTOMERS_P25 TSP_SET_1
50 ORDERS CUSTOMERS_P26 TSP_SET_1
51 ORDERS CUSTOMERS_P27 TSP_SET_1
52 ORDERS CUSTOMERS_P28 TSP_SET_1
53 ORDERS CUSTOMERS_P29 TSP_SET_1
54 ORDERS CUSTOMERS_P30 TSP_SET_1
55 ORDERS CUSTOMERS_P31 TSP_SET_1
56 ORDERS CUSTOMERS_P32 TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
57 ORDERS CUSTOMERS_P33 TSP_SET_1
58 ORDERS CUSTOMERS_P34 TSP_SET_1
59 ORDERS CUSTOMERS_P35 TSP_SET_1
60 ORDERS CUSTOMERS_P36 TSP_SET_1
63 ORDERS CUSTOMERS_P7 C007TSP_SET_1
66 ORDERS CUSTOMERS_P9 C009TSP_SET_1
69 ORDERS CUSTOMERS_P17 C00HTSP_SET_1
72 ORDERS CUSTOMERS_P12 C00CTSP_SET_1
75 ORDERS CUSTOMERS_P14 C00ETSP_SET_1
78 ORDERS CUSTOMERS_P13 C00DTSP_SET_1
81 ORDERS CUSTOMERS_P8 C008TSP_SET_1

ROWNUM TABLE_NAME PARTITION_NAME TABLESPACE_NAME
---------- -------------------- -------------------- --------------------
84 ORDERS CUSTOMERS_P10 C00ATSP_SET_1
87 ORDERS CUSTOMERS_P11 C00BTSP_SET_1
90 ORDERS CUSTOMERS_P16 C00GTSP_SET_1
93 ORDERS CUSTOMERS_P6 C006TSP_SET_1
96 ORDERS CUSTOMERS_P1 C001TSP_SET_1
99 ORDERS CUSTOMERS_P15 C00FTSP_SET_1
102 ORDERS CUSTOMERS_P3 C003TSP_SET_1
105 ORDERS CUSTOMERS_P5 C005TSP_SET_1
108 ORDERS CUSTOMERS_P2 C002TSP_SET_1

108 rows selected.

总共三个表，每个表36个chunks,chunk的个数是在create shardcatalog时指定的

验证chunk是否均匀分布
连接shard catalog数据库
[oracle@shardcat admin]$ sqlplus / as sysdba
SELECT a.name Shard, COUNT(b.chunk_number) Number_of_Chunks
FROM gsmadmin_internal.database a, gsmadmin_internal.chunk_loc b
WHERE a.database_num=b.database_num
GROUP BY a.name
ORDER BY a.name;

SHARD NUMBER_OF_CHUNKS
------------------------------ ----------------
sh1 18
sh2 18

每个表36个chunk，均匀分布在两个shard库上；这也表示一个shard库挂了，那么整个库也就挂了。

用户访问

查看GSM
GDSCTL>config gsm
Name Region ENDPOINT
---- ------ --------
gsmdrt1 region1 (ADDRESS=(HOST=shardgsm)(PORT=1526)(PROTOCOL=tcp))

[oracle@shardgsm ~]$ lsnrctl status gsmdrt1

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 10-DEC-2017 17:46:10

Copyright (c) 1991, 2016, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(HOST=shardgsm)(PORT=1526)(PROTOCOL=tcp))(CONNECT_DATA=(SERVICE_NAME=GDS$CATALOG.oradbcloud)))
STATUS of the LISTENER
------------------------
Alias GSMDRT1
Version TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date 10-DEC-2017 09:56:50
Uptime 0 days 7 hr. 49 min. 21 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.2.0/db_1/network/admin/gsm.ora
Listener Log File /u01/app/oracle/diag/gsm/shardgsm/gsmdrt1/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=shardgsm)(PORT=1526)))
Services Summary...
Service "GDS$CATALOG.oradbcloud" has 1 instance(s).
Instance "catdb", status READY, has 1 handler(s) for this service...
Service "_MONITOR" has 1 instance(s).
Instance "GSMDRT1", status READY, has 1 handler(s) for this service...
Service "_PINGER" has 1 instance(s).
Instance "GSMDRT1", status READY, has 1 handler(s) for this service...
The command completed successfully

添加服务
The oltp_rw_srvc globalservice is a global data service that a client can use to connect to the sharded database. The oltp_rw_srvc service runs the OLTP transactions on theprimary shards. Likewise, the oltp_ro_srvc global service is created to runread-only workload on the standby shards.
本例中只有主，没有备
[oracle@shardgsm ~]$ gdsctl
GDSCTL>add service -service readwrite -role primary
GDSCTL>config service

Name Network name Pool Started Preferred all
---- ------------ ---- ------- -------------
readwrite readwrite.sdbname.oradbcloud sdbname No Yes
GDSCTL>start service -service readwrite

[oracle@shardgsm ~]$ lsnrctl status gsmdrt1

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 10-DEC-2017 18:06:01

Copyright (c) 1991, 2016, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(HOST=shardgsm)(PORT=1526)(PROTOCOL=tcp))(CONNECT_DATA=(SERVICE_NAME=GDS$CATALOG.oradbcloud)))
STATUS of the LISTENER
------------------------
Alias GSMDRT1
Version TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date 10-DEC-2017 09:56:50
Uptime 0 days 8 hr. 9 min. 11 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/12.2.0/db_1/network/admin/gsm.ora
Listener Log File /u01/app/oracle/diag/gsm/shardgsm/gsmdrt1/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=shardgsm)(PORT=1526)))
Services Summary...
Service "GDS$CATALOG.oradbcloud" has 1 instance(s).
Instance "catdb", status READY, has 1 handler(s) for this service...
Service "_MONITOR" has 1 instance(s).
Instance "GSMDRT1", status READY, has 1 handler(s) for this service...
Service "_PINGER" has 1 instance(s).
Instance "GSMDRT1", status READY, has 1 handler(s) for this service...
Service "readwrite.sdbname.oradbcloud" has 2 instance(s).
Instance "sdbname%1", status READY, has 1 handler(s) for this service...
Instance "sdbname%11", status READY, has 1 handler(s) for this service...
The command completed successfully

客户端TNS
schtns =
(DESCRIPTION=
(FAILOVER=on)
(ADDRESS_LIST=
(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL = TCP)(host=shardgsm)(port=1526)))
(CONNECT_DATA=
(SERVICE_NAME=readwrite.sdbname.oradbcloud)
(REGION=region1)
)
)

连接测试
[oracle@shardgsm admin]$ sqlplus sharduser/sharduser@schtns

SQL*Plus: Release 12.2.0.1.0 Production on Sun Dec 10 18:09:28 2017

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Last Successful login time: Sun Dec 10 2017 14:45:39 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL>
SQL> show parameter name

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
_cloud_name string oradbcloud
_dbpool_name string sdbname
_region_name string region1
_shardgroup_name string primary_shardgroup
_shardspace_name string shardspaceora
cdb_cluster_name string sh1
cell_offloadgroup_name string
db_file_name_convert string *, /u01/app/oracle/oradata/SH1
/datafile/
db_name string sh1
db_unique_name string sh1

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
global_names boolean FALSE
instance_name string sh1
lock_name_space string
log_file_name_convert string
pdb_file_name_convert string
processor_group_name string
service_names string sh1

插入数据

insert into customers(custid,firstname,lastname) values(101,'Pengfei','Tan');
insert into customers(custid,firstname,lastname) values(102,'aaaa','bbbb');
insert into customers(custid,firstname,lastname) values(103,'cccc','bbbb');
insert into customers(custid,firstname,lastname) values(104,'dddd','bbbb');
commit;

连接配置库查询
使用PLSQL连接配置库查询

登录服务器查询
[oracle@shardgsm admin]$ sqlplus sharduser/sharduser@catdbtns
col custid for a12
col firstname for a12
col lastname for a12
select custid,firstname,lastname from customers;

CUSTID FIRSTNAME LASTNAME
------------ ------------ ------------
101 Pengfei Tan
102 aaaa bbbb
104 dddd bbbb
103 cccc bbbb

Elapsed: 00:00:02.74
就算是第二次查询，速度依然很慢。
通过GSM查询
[oracle@shardgsm admin]$ sqlplus sharduser/sharduser@schtns
col custid for a12
col firstname for a12
col lastname for a12
select custid,firstname,lastname from customers;

CUSTID FIRSTNAME LASTNAME
------------ ------------ ------------
103 cccc bbbb

查看连接所在分片
SQL> show parameter name

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
_cloud_name string oradbcloud
_dbpool_name string sdbname
_region_name string region1
_shardgroup_name string primary_shardgroup
_shardspace_name string shardspaceora
cdb_cluster_name string sh2
cell_offloadgroup_name string
db_file_name_convert string *, /u01/app/oracle/oradata/SH2
/datafile/
db_name string sh2
db_unique_name string sh2

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
global_names boolean FALSE
instance_name string sh2
lock_name_space string
log_file_name_convert string
pdb_file_name_convert string
processor_group_name string
service_names string sh2
上一次连接GSM连接的分片是SH1，而这次连接的是SH2

在某个分片库上查询
上一步是在SH2上查询，本次在SH1上查询，预计应该有三条数据
[oracle@shard1 ~]$ sqlplus sharduser/sharduser
col custid for a12
col firstname for a12
col lastname for a12
select custid,firstname,lastname from customers;

CUSTID FIRSTNAME LASTNAME
------------ ------------ ------------
101 Pengfei Tan
102 aaaa bbbb
104 dddd bbbb

总共插入四条记录，三个在sh1上，一条在sh2上；通过gsm不指定条件连接则仅是查询某个分库片的数据。

重启

关闭

[oracle@shardgsm admin]$ gdsctl
GDSCTL>stop gsm -gsm gsmdrt1
GSM is stopped successfully

[oracle@shard1 ~]$ schagent -stop
[oracle@shard1 ~]$ lsnrctl stop
SQL> shutdown immediate
[oracle@shard2 ~]$ schagent -stop
[oracle@shard2 ~]$ lsnrctl stop
SQL> shutdown immediate

[oracle@shardcat ~]$ lsnrctl stop
[oracle@shardcat ~]$ sqlplus / as sysdba
SQL> shutdown immediate

启动

[root@shardcat ~]# su - oracle
Last login: Sat Dec 9 21:31:01 CST 2017 on pts/0
[oracle@shardcat ~]$ sqlplus / as sysdba
SQL> startup
[oracle@shardcat ~]$ lsnrctl start

[oracle@shard1 ~]$ sqlplus / as sysdba
SQL> startup
[oracle@shard1 ~]$ lsnrctl start
[oracle@shard1 ~]$ schagent -start

[oracle@shard2 ~]$ sqlplus / as sysdba
SQL> startup
[oracle@shard2 ~]$ lsnrctl start
[oracle@shard2 ~]$ schagent -start

[oracle@shardgsm ~]$ gdsctl
Current GSM is set to GSMDRT1
GDSCTL>start gsm -gsm gsmdrt1
GSM is started successfully
GDSCTL>connect gsm/Admin123@catdbtns
GDSCTL>config shard
Catalog connection is established
Name Shard Group Status State Region Availability
---- ----------- ------ ----- ------ ------------
sh1 primary_shardgroup Ok Deployed region1 ONLINE
sh2 primary_shardgroup Ok Deployed region1 ONLINE

GDSCTL>config service

Name Network name Pool Started Preferred all
---- ------------ ---- ------- -------------
readwrite readwrite.sdbname.oradbcloud sdbname Yes Yes

GDSCTL>databases;
Database: "sh1" Registered: Y State: Ok ONS: N. Role: PRIMARY Instances: 1 Region: region1
Service: "readwrite" Globally started: Y Started: N
Scan: Y Enabled: Y Preferred: Y
Registered instances:
sdbname%1
Database: "sh2" Registered: Y State: Ok ONS: N. Role: PRIMARY Instances: 1 Region: region1
Service: "readwrite" Globally started: Y Started: N
Scan: Y Enabled: Y Preferred: Y
Registered instances:
sdbname%11

连接测试
[oracle@shardgsm ~]$ sqlplus sharduser/sharduser@catdbtns

SQL*Plus: Release 12.2.0.1.0 Production on Sun Dec 10 19:18:02 2017

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Last Successful login time: Sun Dec 10 2017 19:17:56 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
[oracle@shardgsm ~]$
[oracle@shardgsm ~]$
[oracle@shardgsm ~]$ sqlplus sharduser/sharduser@schtns

SQL*Plus: Release 12.2.0.1.0 Production on Sun Dec 10 19:18:07 2017

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Last Successful login time: Sun Dec 10 2017 18:48:54 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> show parameter name

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
_cloud_name string oradbcloud
_dbpool_name string sdbname
_region_name string region1
_shardgroup_name string primary_shardgroup
_shardspace_name string shardspaceora
cdb_cluster_name string sh1
cell_offloadgroup_name string
db_file_name_convert string *, /u01/app/oracle/oradata/SH1
/datafile/
db_name string sh1
db_unique_name string sh1

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
global_names boolean FALSE
instance_name string sh1
lock_name_space string
log_file_name_convert string
pdb_file_name_convert string
processor_group_name string
service_names string sh1
SQL>